Skip to content

LLM tracking

Here we'll cover how to automatically track events from LLM models in Node.js. Events such as prompts, generations and tool calls can be captured with just a few lines of code.

Initialization

Trubrics must be initialized with your LLM client. This helps make the automatic tracking more accurate.

Foundation models

OpenAI must be imported alongside AzureOpenAI:

import { Trubrics } from "@trubrics/trubrics";
import { OpenAI, AzureOpenAI } from 'openai';

export const trubrics = new Trubrics({ 
    apiKey: TRUBRICS_API_KEY,
    openAI: OpenAI
});
// Coming soon
// Coming soon
// Coming soon
// Coming soon
// Coming soon
// Coming soon
// Coming soon
// Coming soon
// Coming soon
import { Trubrics } from "@trubrics/trubrics";
import { OpenAI } from 'openai';

export const trubrics = new Trubrics({ 
    apiKey: TRUBRICS_API_KEY,
    openAI: OpenAI
});
// Coming soon
// Coming soon
// Coming soon

Frameworks

Your LLM client must be imported alongside other LangChain imports:

import { Trubrics } from "@trubrics/trubrics";
import { ChatOpenAI, OpenAIClient } from "@langchain/openai";

export const trubrics = new Trubrics({ 
    apiKey: TRUBRICS_API_KEY,
    openAI: OpenAIClient
});
// Coming soon

Tracking

You can now start tracking prompts, generations and tool calls by using the withProperties wrapper.

The withProperties function wraps around your LLM function and returns the same response object as your LLM function. It takes an optional properties dictionary, which allows you to add context to your LLM events such as user ID's and thread ID's.

const properties = { $user_id: "my-user", $thread_id: "1532ds-243kj-3538", custom_property: "custom_value" };

const llmFunction = (messages) => openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages
});

const completion = await trubrics.withProperties(properties, () => llmFunction(messages));

// Properties can be null
const completionWithoutProperties = await trubrics.withProperties(null, () => llmFunction(messages));

Trubrics properties

Trubrics properties must be prefixed with dollar signs.