Configuring Bedrock as your LLM with llamindex-TS
Marcos Henrique

Marcos Henrique @wakeupmh

About: "Programming isn't about what you know; it's about what you can figure out.” Learning in Public 🧑🏻‍💻

Location:
São José dos Campos
Joined:
May 21, 2019

Configuring Bedrock as your LLM with llamindex-TS

Publish Date: Jul 24 '24
4 1

llamindex for TypeScript is in its early stages compared to the Python version. Still, it already offers great flexibility if you don't want to change the language of your Proof of Concept or your project (honestly, I haven't made anything production-ready yet, so DYR).

You will need to use the package @llamaindex/community because natively, we can't use Bedrock in the llamindex core package, so for this purpose, install @llamaindex/community, it is a package managed by the llamaindex team.
Basically, it's a secondary distribution as it seems to be a way to externalise the open-source contribution (dunno why, but ok)

Cut to the chase

bouncing head man

import { BEDROCK_MODELS, Bedrock } from '@llamaindex/community'
 Settings.llm = new Bedrock({
    model: BEDROCK_MODELS.ANTHROPIC_CLAUDE_3_HAIKU,
 })
Enter fullscreen mode Exit fullscreen mode

I discovered this handy tip while reviewing a pull request's readme. It took some effort to figure it out, so I hope it's helpful to you.

Comments 1 total

Add comment