This page is a work in progress as I add demonstrations of AI with IBM Power Systems. Check back regularly for progress and new demonstrations.
- LLM & RAG Installation, Walkthrough & Demonstration
- CSI Share Five AI Use Cases
- Watsonx & Banking with Power Virtual Server Use Case
- IBM Redbook Question & Answer
- Presentation Search with Slide Finder
- Wallaroo MLOps Platform Demo
- Deploy RAG Workflows with MongoDB Vector Database to IBM Power10
LLM & RAG Installation, Walkthrough & Demonstration
Deploy secure, private AI applications with complete data control
Overview
Learn to build and deploy enterprise-grade Large Language Model (LLM) and Retrieval-Augmented Generation (RAG) applications using OpenShift and IBM Power. This demonstration showcases a complete private AI solution maintaining full data privacy without external API dependencies.
Technology Stack
Core Components: OpenShift container platform, LLama-cpp inference engine, Milvus vector database, Streamlit web interface, with Minio storage and etcd coordination.
Key Features
🔒 Complete Privacy – On-premises deployment with zero external data sharing
📚 Advanced RAG – Custom document processing with vector search capabilities
🎭 Persona-Based AI – Configurable personalities for specialized interactions
⚡ Power10 Optimized – 16-30 tokens/second without GPU requirements
Business Benefits
– Cost Efficient: No subscription fees or ongoing cloud costs
– Secure & Compliant: Full data sovereignty and regulatory compliance
– Customizable: Complete control over AI behavior and responses
Get Started
📋 Complete Lab Guide: https://nas01.tallpaul.net/wordpress/llm-rag-lab
Available for live demonstrations and custom POC/MVP development assistance.
Deploy private AI for your enterprise with open-source technologies and maintain complete control over your data.
CSI Share Five AI Use Cases
This demonstration showcases cutting-edge AI workloads running on IBM Power10 infrastructure, highlighting the platform’s capabilities for enterprise AI applications. The setup features a robust OpenShift cluster deployment with three master nodes and five worker nodes, with AI workloads primarily executing on dedicated Power10 hardware.
The demo presents several practical AI use cases that organizations can implement on-premises. Key demonstrations include a custom question-and-answer system that processes uploaded PDF documents (demonstrated with IBM’s new Power S1012 system documentation), providing intelligent responses about technical specifications and capabilities. Additional use cases feature PII data extraction and redaction capabilities, automated marketing content generation that creates strategic briefs from business objectives, email summarization for rapid communication processing, and sales conversation analysis that extracts key insights and topics from customer interactions.
The platform demonstrates flexibility by supporting multiple large language models, including Google’s FLAN, with the ability to integrate models from Hugging Face and NVIDIA as needed. All processing occurs on-premises, ensuring data privacy and security. The system delivers rapid inference capabilities, making it suitable for real-time business applications while leveraging Power10’s performance advantages for AI workloads.
Watsonx & Banking with Power Virtual Server Use Case
Accelerating Business Intelligence with IBM Power Virtual Server and Watsonx.ai Integration
This demonstration showcases how organizations can leverage the powerful combination of IBM Power Virtual Server (PowerVS) and Watsonx.ai to transform business operations through intelligent automation. The integration enables users across different roles to access critical data insights without requiring specialized technical skills.
The demo features two practical banking scenarios that illustrate real-world applications. First, a fraud investigator demonstrates how natural language prompts can be converted into SQL queries through Watsonx.ai’s foundation models running on IBM Cloud. This seamless integration allows investigators to query suspicious transactions directly from their core database on PowerVS, streamlining fraud detection processes.
The second scenario follows a marketing manager identifying high-value clients for targeted campaigns. Using the same intuitive interface, complex database queries are automatically generated and executed, presenting results in easily digestible formats.
Key benefits highlighted include eliminating the need for specialized IT or AI expertise, reducing time-to-insight, and enabling business users to make data-driven decisions independently. The PowerVS GenAI Assistant serves as an intelligent intermediary, translating business questions into technical queries while maintaining secure access to enterprise data systems.
This integration demonstrates how modern AI can democratize data access while preserving security and performance standards.
IBM Redbook Question & Answer
Presentation Search with Slide Finder
Wallaroo MLOps Platform
🚀 Learn how to deploy AI models on IBM Power10 architecture with just ONE extra parameter!
In this comprehensive demo, we walk through Wallaroo’s streamlined 3-step workflow to configure and deploy AI models optimized for IBM’s Power10 architecture. Whether you’re working in the cloud, on-premises, or at the edge, this tutorial shows you exactly how to leverage high-performance scalable processing for your data-intensive AI workloads.
What You’ll Learn:
✅ How to deploy a quantized Llama 3 LLM with custom RAG implementation
✅ Setting up computer vision models for retail edge deployment
✅ Configuring IBM Power10 optimization with a single parameter
✅ Extending models to edge devices with containerized deployment
✅ Monitoring and observability through Wallaroo’s operations center
✅ Centralized logging and automated drift detection
Key Features Demonstrated:
- Wallaroo’s flexible custom framework
- Resource-efficient CPU-only deployment
- Edge bundle creation for distributed systems
- Real-time pipeline monitoring and insights
- Seamless cloud-to-edge model extension
Perfect for ML engineers, DevOps teams, and AI practitioners looking to optimize their model deployment workflows on enterprise-grade IBM Power10 infrastructure.
Timestamps:
0:00 – Introduction to Wallaroo + IBM Power10
0:40 – Uploading and configuring the LLM
1:30 – Deployment and testing
2:30 – Operations center walkthrough
3:20 – Edge deployment demo
4:30 – Centralized monitoring and insights
Deploy RAG Workflows with MongoDB Vector Database to IBM Power10
Description: In this technical demonstration, we show how Wallaroo.AI’s custom framework makes it seamless to deploy sophisticated RAG (Retrieval-Augmented Generation) workflows to on-premises IBM Power10 infrastructure.
🔧 What You’ll Learn:
- How to implement a RAG pipeline using Wallaroo’s flexible custom framework
- Deploying AI models optimized for IBM Power10 architecture
- Integrating with on-premises MongoDB vector databases
- Creating self-hosted API endpoints for LLM-driven applications
- Containerizing and publishing AI pipelines with just a few lines of code
⚡ Key Features Demonstrated:
- Environment-agnostic pipeline development
- Automatic optimization for Power10 hardware
- OCI-compliant containerization (Docker/Podman support)
- Real-time inference with personalized responses
- Built-in monitoring and feedback loops to Wallaroo’s Operations Center
This demo showcases a complete end-to-end workflow from development to production deployment, highlighting how enterprises can leverage their existing on-premises infrastructure while maintaining the flexibility and ease of cloud-native AI development.
Perfect for data scientists, ML engineers, and IT professionals exploring enterprise AI deployment strategies on IBM Power systems.
Leave a Reply