Brev for Smol

Brev ❤️ Smol

How to run & deploy Falcon:

1. Use the Falcon model template by clicking the button below:

This will spin up an A100 running falcon

2. SSH into the instance

Run the following command to open VS Code into your instance

brev open falcon-api

Note: this can take a while since it's downloading the model and all dependencies

3. Run the server

python app.py

You can make sure it's working by curling

curl -X POST -H "Content-Type: application/json" -d '{"text":"Once upon a time"}' http://localhost:8888/predict

4. Expose via Tunnels

Expose your service via (tunnels)[https://brev.dev/docs/how-to/brev-tunnel].

Note: the Tunnels feature is now called Deployments. Docs will be updated shortly!

5. Text Nader if there's anything 🤙

How to run & deploy mpt:

Coming soon

How to run & deploy redpajama:

Coming soon