LLM Endpoints Lab
Familiarize yourself with Anyscale Endpoints, a service similar to the OpenAI API but with open models.
This lab is an opportunity to familiarize yourself with
If you’re new to LLM applications, focus on the intro level activity. If you’ve worked with LLM apps before, you may have time to try the additional activities.
The main activity in this lab is to duplicate the Intro_LLM_Apps notebook, and
https://ray-summit-training-jrvwy.cld-kvedzwag2qa8i5bj.s.anyscaleuserdata-staging.com/v1 as the API base URL
max_tokens=100 to the
create() call to get shorter faster answers
Notice that some of the model behavior is a bit different from the GPT-3.5-Turbo version
Try to change the prompts in order to get this model to achieve the same level of functionality as GPT-3.5-Turbo
Alter the email tool/agent functionality to also include the destination email address (since sending a real email would of course require the target address).
Hint: this will require altering the “database” as well as the prompts