Okahu’s key value proposition is AI Observability as a Service, offering proactive monitoring of AI applications and their underlying cloud infrastructure to improve reliability, performance, and cost-effectiveness. Okahu specifically targets AI workloads with generative capabilities leveraging various platforms. The primary objective of this project was to extend these capabilities to the AWS ecosystem, utilizing the serverless GenAI platform in Bedrock and models deployed in SageMaker. The project aimed to explore and implement these capabilities in two phases: first with AWS Bedrock, followed by running models in SageMaker.

The overarching goal was to develop reference implementations for a chatbot application:

• AWS Bedrock: Leverage AWS Bedrock, a serverless application platform, to host the chatbot service.
• Amazon SageMaker: Utilize Amazon SageMaker, a machine learning platform, for model hosting.

The initial scope of the project focused on the first two reference implementations:

• AWS Bedrock: The primary objective was to understand how to refactor the Okahu chatbot service to run on AWS Bedrock while closely monitoring its performance.
• Amazon SageMaker: The focus was to host a representative chatbot application trained on data from a Git repository in an AWS environment. This application would be hosted on Elastic Beanstalk, a fully managed platform for deploying and scaling web applications.

Key elements of the solution included:

• Infrastructure as Code: By providing the deliverables as infrastructure as code, SnapSoft ensured reproducibility and clear documentation, reducing the risk of configuration errors and simplifying future updates.
• AWS Funding: SnapSoft secured funding from Amazon to cover part of the project costs, easing the financial burden on Okahu.
• Comprehensive Onboarding and Communication: The project featured multiple onboarding sessions and continuous communication, ensuring alignment on goals and expectations. This included working within Okahu’s Amazon environment and providing regular updates through a well-documented repository.

The solution leveraged a serverless application to support Titan-based embedding and the OpenSearch vector database. For query retrieval, users can choose a Large Language Model (LLM) from the Bedrock repository using Model Access. For traceability, monitoring data will be collected through CloudWatch, which allows for the instrumentation of application code to monitor API calls and service utilization (tokens, GPU/CPU).

The project concluded successfully within the stipulated four-week period. Okahu launched their marketplace offerings on AWS, achieving their goals of platform integration and market readiness.

Key outcomes included:

• Improved Deployment Processes: Okahu gained a clear understanding of deploying AI applications using well-architected frameworks and reference architectures on AWS.
• Enhanced Performance and Cost Efficiency: The implemented solution improved the performance of AI-driven applications, reducing response times and operational costs.
• Effective Coordination and Communication: The project's success was attributed to SnapSoft's meticulous planning, clear documentation, and robust communication strategy, which ensured all stakeholders were aligned and any potential issues were promptly addressed.

Overall, the collaboration with SnapSoft enabled Okahu to achieve their strategic objectives, setting a solid foundation for future cloud-based AI deployments. The reference implementations provided a comprehensive understanding of the options available for deploying and hosting a chatbot application on AWS, helping Okahu make informed decisions based on application requirements, performance, and operational preferences.

Learn how SnapSoft can streamline your AWS integrations and drive strategic success for your business. For more information, visit SnapSoft.