Over the last year and a half, AI has moved from research labs into everyday use. Advanced language models have made AI-powered app features possible, but developing and refining these features beyond a prototype is challenging. Many are still working on how to implement these features at scale and evaluate their performance for continuous improvement. Ensuring safety and stability throughout this process adds another layer of complexity. The reality is, everyone could use some help.

Today we’re excited to introduce Firebase Genkit, now available in beta. Genkit is a new open source framework built for JavaScript/TypeScript developers (with Go coming soon) for creating Node.js backends to help you integrate AI-powered features into your new or existing apps using large language models and related services. Genkit lets you integrate diverse data sources, models, cloud services, agents, and more with the code-centric approach you are already familiar with.

Genkit is a developer-first framework for building AI-powered features for your apps

Genkit offers a rich AI-centric local developer tooling making building and debugging your AI workloads easier. Once you’re ready to go to production, you can also use Genkit to deploy your solution to Firebase or Google Cloud, and monitor your app to ensure it is production-ready.

Here is how Genkit makes it easier to develop AI features for your apps:

1. Intuitive for developers
2. AI focused local development
3. High performance inference
4. Open and extensible
5. Ready for serverless
6. Built for apps in production

Learning new technologies takes time and effort, especially when it comes to AI with rapidly evolving concepts and approaches. We believe AI frameworks should reduce complexity, not add to it. That is why we designed Genkit to feel familiar and intuitive with a minimum learning curve to get started.

configureGenkit({
  plugins: [
    googleAI(),
  ],
  logLevel: "debug",
  enableTracingAndMetrics: true,
});

export const menuSuggestionFlow = defineFlow(
  {
    name: "menuSuggestionFlow",
    inputSchema: z.string(),
    outputSchema: z.string(),
  },
  async (subject) => {
    const prompt =
      `Suggest an item for the menu of a ${subject} themed restaurant`;

    const llmResponse = await generate({
      model: geminiPro,
      prompt: prompt,
      config: {
        temperature: 1,
      },
    });

    return llmResponse.text();
  }
);

Our first release is built for JavaScript/TypeScript developers, making building AI-powered apps available to Node.js backend developers.

Just like you’d expect, install Genkit by typing npm i -g genkit into your terminal, then run genkit init to get your new project started.

Genkit’s first release has AI specific integrations from Google and support for some popular 3rd party open source projects, including:

• Large Language Models such as Google’s Gemini and open source models via Ollama
• Vector Databases like Chroma, Pinecone, Cloud Firestore, and PostgreSQL (pgvector)
• Embedding Services from Google (Google AI & Vertex)
• Evaluators like Vertex AI’s rapid evaluation API

This initial set of capabilities is just a glimpse of what’s to come. We have an exciting roadmap of features and enhancements.

Building AI-powered apps introduces unique challenges. Alongside traditional debugging, you’ll need to master new concepts like crafting effective prompts for large language models, monitoring outputs from vector stores, and eventually deploying your project to testing or production environments.

To assist you with these challenges and streamline deployment, we’ve built a delightful developer tooling including a browser-based UI and a feature-rich CLI.

With these tools you can:

• Easily get started by initializing your project and kicking off a debug session
• Experiment with AI components like model prompts, indexers, retrievers, and tools. This enables you to locally execute AI features end-to-end and quickly test and iterate on outcomes.
• View complete traces and metadata for every model and component interaction for full observability and efficient debugging.

If you like working within VS Code or Project IDX, you can even open the Genkit Developer UI in the VS Code integrated browser and use it side-by-side with your code.

Working with open models such as Google’s Gemma is also possible thanks to the Genkit plugin for Ollama. When you combine Genkit and Ollama together, you can host open models locally on your machine for development or even deployment to your own custom environment.

Recently a Google collaboration with NVIDIA made working with Gemma even better when running on local NVIDIA GPUs such as RTX series. Thanks to these optimizations you’ll see improved inference performance when working with Gemma. You can learn more on the NVIDIA blog by reading through the tutorial on how to get started.

Once you’ve finished building and debugging your project locally, you’ll need to deploy it and monitor it in your test and eventually production environments to make sure it’s serving your users like you expected. To help with this, Genkit is built from the ground up as a cloud-native framework with robust instrumentation out-of-the-box for tracing and logging at scale.

For example, Genkit offers plugins for Google Cloud and Firebase that provide seamless cloud-native integration. The Google Cloud plugins lets you easily export logs, metrics, and traces from your AI-powered features to Cloud Logging, Cloud Tracing. You can then deploy your features to Cloud Run or as an Express server with with a few simple CLI commands.

While we provide the above integrations out-of-the-box, Genkit’s instrumentation is implemented through the widely used OpenTelemetry standard and can also be integrated with many popular observability platforms.

We also provide a Firebase plugin that enables you to deploy to Cloud Functions for Firebase, secured with Firebase Authentication and Firebase App Check. No more sleepless nights, worrying that someone might steal your API keys or thrash your AI endpoints. This plugin also allows you to persist traces in Firestore for future debugging or evaluation.

We know developers love open source because of the transparency, freedom, and community it provides – and that’s why Genkit’s framework is open source, available on GitHub under the Apache 2.0 license. As we iterate on and expand the framework, we’ll be building in the open so you can keep tabs on our progress, make suggestions, easily file issues, and even contribute code!

Genkit is also designed to be open to any and all models, vector stores, embedders, evaluators, and other components through its plugin system. Genkit provides pre-built plugins that give you easy access to models and vector stores built by Google and popular third parties as highlighted above.

Follow our guide to learn how to build your own plugins for Genkit, and contribute them to the community.

Using Genkit with your preferred large language model and supporting services, you can create countless unique experiences for your customers, for example:

• Intelligent Agents: Create AI-powered agents that can understand complex requests and use tools to execute tasks like booking flights and hotels based on your preferences, or make dinner reservations at your favorite restaurant.
• Customer Support: Streamline customer support with AI that leverages your company’s knowledge base and documentation to provide faster, more accurate responses to inquiries.
• Multimedia Content Creation: Generate engaging multimedia content tailored to your needs such as in-game character designs and dynamic dialogue.
• Coding Assistants: Accelerate software development with AI-powered assistants that can generate, review, and and refine code.
• Language Translation: Break down language barriers with AI-driven translation services. This is particularly valuable for travel apps that can translate local menus, signs, or conversations, enhancing the user experience for international travelers.
• Build robust AI Evaluation: Ensure the quality and effectiveness of your generative AI features by developing custom evaluation metrics or leveraging existing frameworks like Vertex Rapid Evaluation.

These are just some of the examples we’ve seen in the community, the possibilities are endless.

Now that Genkit is available in public beta, you can start using it to add AI workflows to your apps. Of course we won’t stop here, with support for more languages and frameworks to follow.

Here’s where to get started:

• Visit our GitHub repo
• Take a look at our documentation

We can’t wait to see what you build with Genkit!

Please note that while Genkit is in beta we’ll continue to make changes to build the best possible framework, including possible breaking changes in the API design. This will continue until we release our 1.0.0 stable build later this year.