diff --git a/README.md b/README.md index 9c51ae4..305aebb 100644 --- a/README.md +++ b/README.md @@ -10,16 +10,14 @@ View this on the [Tutorial Homepage](https://supercontainers.github.io/sc-tutori ## Details -Full-day Tutorial Session +Half-day Tutorial Session -Venue: Supercomputing Conference (SC 25) +Venue: ISC High Performance 2026 -Location: St. Louis, MO, USA +Location: Hamburg, Germany -Link: SC 2025 Tutorial Details](https://sc25.conference-program.com/presentation/?id=tut109&sess=sess250) - -Keywords: Containerized HPC, System Software and Runtime Systems, Scientific Software Development, DevOps +Keywords: Containerized HPC, System Software and Runtime Systems, Scientific Software Development, DevOps, AI ## Abstract @@ -35,11 +33,8 @@ Please consult the website for prerequisites and recommended setup steps. ## Questions -You can ask questions verbally or with this [Google Doc](https://docs.google.com/document/d/11gMZ-T7iA5XiRWPLYIqX7Gqv7RMb-NF9kzGYHrnOi04/edit?usp=sharing). -Please append your question below the others in the document. - -We have also created a Slack Team for this. The invitation link is [here](https://join.slack.com/t/hpc-containers/shared_invite/enQtODI3NzY1NDU4OTk5LTUxOTgyOWJmYjIwOWI5YWU2MzBhZDI3Zjc1YmZmMjAxZjgzYzk4ZWEwNmFlNzlkOWI0MGNlZDNlMTBhYTBlOWY). - - -## Schedule - Autogenerated from the metadata + +You can ask questions verbally or with this [Google Doc](). Please append your question below the others in the document. + +We have also created a Slack Team for this. The invitation link is [here](https://join.slack.com/t/hpc-containers/shared_invite/enQtODI3NzY1NDU4OTk5LTUxOTgyOWJmYjIwOWI5YWU2MzBhZDI3Zjc1YmZmMjAxZjgzYzk4ZWEwNmFlNzlkOWI0MGNlZDNlMTBhYTBlOWY). diff --git a/_episodes/ai.md b/_episodes/ai.md new file mode 100644 index 0000000..47c938f --- /dev/null +++ b/_episodes/ai.md @@ -0,0 +1,98 @@ +--- +title: "Agentic AI" +teaching: 15 +exercises: 20 +questions: +- How do I quickly develop and deploy agents in Kubernetes? +- How do multi-agent systems communicate? +objectives: +- Learn about developing AI agents +- Learn about connecting AI agents to MCP servers +- Learn about deploying AI systems to Kubernetes +--- + +# Agentic AI + +## Currency Agent + +```bash +git clone https://github.com/srbdev/adk_currency_agent.git +cd adk_currency_agent +cp .env.example .env +vim .env +``` + +1. Update `OPENAI_API_BASE` with `https://openrouter.ai/api/v1` +2. Update `OPENAI_API_KEY` with your key + +```bash +uv sync +uv run currency_agent +``` + +## Client + +```bash +git clone https://github.com/srbdev/currency_client.git +cd currency_client +uv venv +source .venv/bin/activate +uv pip install -r requirements + +uv run python cli.py --url http://localhost:10999 +``` + +Enter a user prompt to make sure that the backend agent is correctly bootstrap with the LLM. + +## Kubernetes + +```bash +kubectl create namespace NAME +kubectl apply -f deployment.yaml -n NAME + +kubectl port-forward svc/currency-agent-service 10999:10999 -n NAME +``` + +1. Restart client + +## MCP + +```bash +git clone https://github.com/srbdev/currency_mcp.git +cd currency_mcp +kubectl apply -f deployment.yaml -n NAME +``` + +### Update Agent + +```bash +git checkout with-remote-mcp + +# If `git` complains about overwritting changes in `deployment.yaml` +git stash +git checkout with-remote-mcp +git stash pop +# make sure no conflicts in `deployment.yaml` +``` + +1. Update `MCP_SERVER_HTTP_URL` with Service IP address from MCP server + +```bash +kubectl get services -n NAME +# and copy `CLUSTER-IP` for `currency-mcp-service` row + +kubectl delete deployments/currency_agent -n NAME +kubectl apply -f deployment.yaml -n NAME +kubectl port-forward svc/currency-agent-service 10999:10999 -n NAME +``` + +1. Restart client + + +## Example Prompts + +``` +Find the list of supported EU currencies and show the exchange rate with USD in markdown format +how much is 10 USD in CAD? +how did the EUR value fluctuate against USD from jan 1st, 2026 to now? +``` diff --git a/_episodes/kubernetes.md b/_episodes/kubernetes.md new file mode 100644 index 0000000..c7f230a --- /dev/null +++ b/_episodes/kubernetes.md @@ -0,0 +1,163 @@ +--- +title: "Container services and Kubernetes" +teaching: 25 +exercises: 20 +questions: +- How should services like databases or portals be deployed? +- What is different about deploying services versus HPC applications? +objectives: +- Learn about different frameworks used for deploying services +keypoints: +- Docker Compose is a simple tool to deploy basic services using containers +- Frameworks like Kubernetes, Rancher and OpenShift can help with deploying and managing complex containerized services. +--- + +# Container Services and Kubernetes + +## First Steps + +```bash +kubectl get nodes +kubectl get pods +kubectl get pods -A +``` + +## Create a Namespace + +```bash +kubectl create namespace NAME +``` + +## Deploy the Application + +Create `demo-app.yaml` + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: backend-api +spec: + replicas: 2 + selector: + matchLabels: + app: backend-api + template: + metadata: + labels: + app: backend-api + spec: + containers: + - name: api + image: nginx:alpine + ports: + - containerPort: 80 + resources: + requests: + memory: "64Mi" + cpu: "100m" + limits: + memory: "128Mi" + cpu: "200m" +--- +apiVersion: v1 +kind: Service +metadata: + name: backend-service +spec: + selector: + app: backend-api + ports: + - port: 80 + targetPort: 80 + type: ClusterIP +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: app-ingress +spec: + rules: + - host: demo.local + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: backend-service + port: + number: 80 +``` + +```bash +kubectl apply -f demo-app.yaml -n NAME # OR --namespace=NAME +kubectl get pods,svc,ingress -n NAME +``` + +## Key Kubernetes Features + +### Self-Healing + +```bash +# Show running pods +kubectl get pods -n NAME + +# Kill a pod +kubectl delete pod pod-name-xyz -n NAME + +# Watch Kubernetes automatically recreate it +kubectl get pods -w -n NAME +``` + +### Scaling + +```bash +# Scale up during "traffic spike" +kubectl scale deployment backend-api --replicas=5 -n NAME +kubectl get pods -n NAME + +# Scale down +kubectl scale deployment backend-api --replicas=2 -n NAME +``` + +### Rolling Updates + +```bash +# Update to new version with zero downtime +kubectl set image deployment/backend-api api=nginx:1.21-alpine -n NAME + +# Watch the rolling update +kubectl rollout status deployment/backend-api -n NAME +kubectl get pods -n NAME + +# Rollback if needed +kubectl rollout undo deployment/backend-api -n NAME +``` + +### Resource Management + +```bash +# Show resource usage +kubectl top nodes +kubectl top pods -n NAME +``` + +## Port Forwarding + +```bash +curl http://localhost:8000 + +# Forward local port to the service +kubectl port-forward svc/backend-service 8000:80 -n NAME + +# Access in browser or curl +curl http://localhost:8000 +# or open http://localhost:8000 in browser +``` + +## Clean up Namespaces + +```bash +kubectl delete namespaces NAME +``` diff --git a/_episodes_sc25/.00.template.md b/_episodes_sc25/.00.template.md new file mode 100644 index 0000000..4654c2b --- /dev/null +++ b/_episodes_sc25/.00.template.md @@ -0,0 +1,79 @@ +--- +title: "TEMPLATE EPISODE - TO BE HIDDEN" +teaching: 10 +exercises: 20 +questions: +objectives: +- this one +keypoints: +- that one +--- + + +### Title (header 3 for size reasons) + +#### Subtitle + +##### subsubtitle...and so on + + +Text types: **bold**, *italic*, ***bold+italic***, `code monospace`. + +How to add a new line in same paragraph. +Add two spaces at end of line, then newline. + +Leave an entire blank line for a new paragraph. + +``` +code block +``` + +* itemise + +1. enumerate + +> cite + +I recommend HTML to embed images, so it's customisable: + +Pawsey Logo + +For comments in the markdown source, go again the HTML way with ``. + + + +### Carpentry style: special syntax + +When embedding images or links to files in this repo, you need syntax like in the image example above: `{{ page.root }}/fig/pawsey.jpeg`. +The *page.root* sugar ensures the file is found at the time the HTML page is created. + +Coloured side for code blocks: + +``` +# This is bash +``` +{: .bash} + +After the closing triple ticks, add one of these: `{: .bash}`, `{: .source}`, `{: .output}`, `{: .error}`. There's similar sugar for python, r, sql, make and matlab. +I normally use *bash* for shell commands, *source* for scripts, and *output*/*error* as appropriate. + + +> ## Text box to highlight, *ie* callout, stuff +> +> This is really important, right? +> Note the use of cite blocks `> `, the *Header ##*, and the `{: .callout}` keyword at the end. +{: .callout} + + +> ## Challenge +> +> Do you know this? +> +> > ## Solution +> > +> > Sure! +> {: .solution} +{: .challenge} + + +There are other specials blocks that I normally don't use, in particular `discussion` and `checklist`. diff --git a/_episodes_sc25/.keep b/_episodes_sc25/.keep new file mode 100644 index 0000000..e69de29 diff --git a/_episodes/11.intro.md b/_episodes_sc25/11.intro.md similarity index 100% rename from _episodes/11.intro.md rename to _episodes_sc25/11.intro.md diff --git a/_episodes/12.docker.md b/_episodes_sc25/12.docker.md similarity index 100% rename from _episodes/12.docker.md rename to _episodes_sc25/12.docker.md diff --git a/_episodes/13.best.md b/_episodes_sc25/13.best.md similarity index 100% rename from _episodes/13.best.md rename to _episodes_sc25/13.best.md diff --git a/_episodes/14.intro-hpc.md b/_episodes_sc25/14.intro-hpc.md similarity index 100% rename from _episodes/14.intro-hpc.md rename to _episodes_sc25/14.intro-hpc.md diff --git a/_episodes/15.install.md b/_episodes_sc25/15.install.md similarity index 100% rename from _episodes/15.install.md rename to _episodes_sc25/15.install.md diff --git a/_episodes/16.break.md b/_episodes_sc25/16.break.md similarity index 100% rename from _episodes/16.break.md rename to _episodes_sc25/16.break.md diff --git a/_episodes/22.run-hpc.md b/_episodes_sc25/22.run-hpc.md similarity index 100% rename from _episodes/22.run-hpc.md rename to _episodes_sc25/22.run-hpc.md diff --git a/_episodes/24.qanda.md b/_episodes_sc25/24.qanda.md similarity index 100% rename from _episodes/24.qanda.md rename to _episodes_sc25/24.qanda.md diff --git a/_episodes/25.lunch.md b/_episodes_sc25/25.lunch.md similarity index 100% rename from _episodes/25.lunch.md rename to _episodes_sc25/25.lunch.md diff --git a/_episodes/32.servics.md b/_episodes_sc25/32.servics.md similarity index 100% rename from _episodes/32.servics.md rename to _episodes_sc25/32.servics.md diff --git a/_episodes/35.e4s.md b/_episodes_sc25/35.e4s.md similarity index 100% rename from _episodes/35.e4s.md rename to _episodes_sc25/35.e4s.md diff --git a/_episodes/39.break.md b/_episodes_sc25/39.break.md similarity index 100% rename from _episodes/39.break.md rename to _episodes_sc25/39.break.md diff --git a/_episodes/41.advanced.md b/_episodes_sc25/41.advanced.md similarity index 100% rename from _episodes/41.advanced.md rename to _episodes_sc25/41.advanced.md diff --git a/_episodes/42.stories.md b/_episodes_sc25/42.stories.md similarity index 100% rename from _episodes/42.stories.md rename to _episodes_sc25/42.stories.md diff --git a/_episodes/43.wrapup.md b/_episodes_sc25/43.wrapup.md similarity index 100% rename from _episodes/43.wrapup.md rename to _episodes_sc25/43.wrapup.md diff --git a/index.md b/index.md index ea8d4bf..4982665 100644 --- a/index.md +++ b/index.md @@ -6,33 +6,26 @@ permalink: index.html # Is the only page that don't follow the partner /:path/i -Containers Tutorial Session @ Supercomputing Conference 2025 (SC25) +Containers Tutorial Session @ ISC High Performance 2026 (ISC26) -Date: Sunday, 16 November 2025 8:30am - 5pm CST +Date: Monday, 22 June 2026 2:00PM - 6PM (Europe/Berlin) -Location: St. Louis, MO, USA +Location: Hamburg, Germany -Link: [SC 2025 Tutorial Details](https://sc25.conference-program.com/presentation/?id=tut109&sess=sess250) - -Keywords: Containerized HPC, System Software and Runtime Systems, Scientific Software Development, DevOps +Keywords: Containerized HPC, System Software and Runtime Systems, Scientific Software Development, DevOps, AI > ## Prerequisites > > This is a hands-on tutorial. Participants should bring a laptop. We will use a AWS instances for the main exercises. > -> #### NERSC Perlmutter (optional) -> In addition to the AWS instances, you can also try running some of the exercises on NERSC’s Perlmutter system. -> > See also the [Setup](./setup.html) page. {: .prereq} > ## Questions > -> You can ask questions verbally or with this [Google Doc](https://docs.google.com/document/d/1zrWRGeDEbokQq03hAHxZzXpNzBSp4YSKJhHiNtjSe1Y/edit?usp=sharing). -> Please append your question below the others in the document. +> You can ask questions verbally or with this [Google Doc](). Please append your question below the others in the document. > > We have also created a Slack Team for this. The invitation link is [here](https://join.slack.com/t/hpc-containers/shared_invite/enQtODI3NzY1NDU4OTk5LTUxOTgyOWJmYjIwOWI5YWU2MzBhZDI3Zjc1YmZmMjAxZjgzYzk4ZWEwNmFlNzlkOWI0MGNlZDNlMTBhYTBlOWY). {: .callout} - diff --git a/setup.md b/setup.md index 467c06e..a47d7e6 100644 --- a/setup.md +++ b/setup.md @@ -7,10 +7,10 @@ root: . ### Key requirement -The main requirement for this workshop is a personal computer with a web browser and, oppotionally, a command line shell program. - +The main requirement for this workshop is a personal computer with a web browser and, optionally, a terminal. *Windows* users: [MobaXterm](https://mobaxterm.mobatek.net/download-home-edition.html) is the *preferred option*. The *Portable Edition* is best, as it does not require administrative privileges. + In this way you will be able to follow the online materials and to login to a facility with the required software stack. @@ -24,13 +24,6 @@ This tutorial is supported by the Amazon AWS Machine Learning Research Awards. E After the tutorial, you can boot our tutorial image yourself on Amazon EC2 to run through the tutorial again. We recommend you use your own EC2 key and change the password. -US-West-Oregon: ami-0fe12765123c6a840 - - -### NERSC Training Accounts (optional) - -In addition to the AWS instances, you can also try running some of the exercises on NERSC’s Perlmutter system. Please apply for a NERSC training project membership by following the instructions here: [Form](https://forms.gle/nftFLGkkfjbLzAXt8) (non-NERSC users due Nov 1, and existing NERSC users due Nov 12). Please note that, due to security restrictions, access may be limited but all of the exercises can be done on the AWS instances. - ### Materials repository @@ -39,7 +32,7 @@ To run the examples yourself, you can download the materials with: ```bash cd ~ git clone https://github.com/supercontainers/sc-tutorials.git -cd sc-tutorials/exercies +cd sc-tutorials/exercises ``` @@ -69,7 +62,7 @@ cd sc-tutorials/exercies > ### macOS or Windows machine > > For *Singularity*, you will need to setup a Linux virtual machine, and then follow the same instructions as above. -> It's not as bad as it sounds... the main two options are: +> It's not as bad as it sounds... The main two options are: > - Vagrant: follow these instructions by Sylabs on [Setting up Singularity with Vagrant](https://singularity.hpcng.org/admin-docs/3.5/installation.html#installation-on-windows-or-mac) (*macOS* users: DO NOT use the proposed *Singularity Desktop*, use Vagrant instead); > - Multipass: follow instructions from the [Multipass Homepage](https://multipass.run). >