Enhancing App Development: Integrating GitHub Models with Firebase

Introduction of Shabbi and Open Source Friday

• Welcome to Open Source Friday, a GitHub power sponsor Community-led event that highlights projects, maintainers, contributors, and the ecosystem, and also explores new projects, with today's guest being Shabbi, a super special guest who will present in English today and in Spanish next week 2m14s.
• The host is back in their home office after a bit of forced travel due to hurricanes and weather situations, and is sending love to those affected, and is happy to be back for another edition of Open Source Friday 2m21s.
• Open Source Friday is an event that highlights not only projects but also the amazing maintainers, contributors, and the ecosystem that everyone depends on, and gives an opportunity to explore new projects 2m55s.
• Shabbi is a guest who was met at the GitHub Universe event last year, and is a nice person, open source contributor, maintainer, and creator, making it a great pleasure to have him on the show 4m30s.

Shabbi's Background and Achievements

• Shabbi was born in Valencia, currently lives in Madrid, and has over 7 years of experience in cloud, Kubernetes, DevOps, QA, microservices, conversational AI, and mentoring startups 4m54s.
• Shabbi is part of the Google for Startups program, an organizer of GDG Madrid, Deast Madrid, and the Alexa community in Spanish, and enjoys music, friends, and is the author of a book called The Alexa Revolution 5m17s.
• Shabbi is a Microsoft MVP, GitHub Star, Alexa Champion, Google Developer Expert, Google Cloud Champion Innovator, Machine Learning AWS Community Builder, and a Content Hero 5m37s.

GenKits Project Overview

• The project that Shabbi will be presenting today is called GenKits 6m3s.
• An open-source plugin has been created to integrate GitHub models with Firebase, allowing users to access GitHub models and incorporate them into their applications 6m6s.
• The plugin's creator, Shabi, was inspired to develop the plugin after gaining early access to GitHub models as a GitHub star and being impressed by their ease of use 7m46s.
• Shabi was also influenced by the Yen Kit framework from the Firebase team, which allows for the creation of generative AI apps, but noticed the lack of a GitHub models plugin and decided to build one 8m46s.
• The plugin is fully functional and enables users to create generative AI applications using GitHub models in under five minutes 9m34s.

GenKits Plugin Ecosystem and Development

• The plugin is part of the Yen Kit plugin ecosystem, which allows developers to create their own plugins 9m3s.
• Shabi is sharing their screen to demonstrate the plugin's capabilities and show the available GitHub models 7m58s.
• The plugin's development was rapid, with Shabi starting work on the idea shortly after GitHub models were announced 10m12s.

Firebase Gen Kit Framework

• Firebase Gen Kit is an open-source project and a framework that allows users to create generative AI applications in a super easy way, with a focus on ease of use and developer experience 10m42s.
• Unlike other frameworks like Llama Index and Lang Chain, Firebase Gen Kit is designed to be straightforward and easy to use, without requiring users to learn a complex abstraction layer 11m32s.
• Firebase Gen Kit is not limited to Firebase functions and can be used as a couple of libraries to deploy whatever users want, with support for Node, JavaScript, TypeScript, and Go (in alpha) 12m37s.
• The framework has a plugin ecosystem that allows users to create different types of plugins, connect to different models, and use various LLMs and embedding models 13m19s.
• Firebase Gen Kit has official plugins for Google Cloud, Firebase, and vector databases, and supports building retrieval-augmented generation applications 14m19s.
• The framework also features tool calling, which allows users to define external functions that LLMs can call, covering most of the needs for building a generative AI app 15m16s.

GitHub Models and Integration

• GitHub Models is a marketplace where users can find available models, accessible by going to github.com/marketplace and clicking on models 15m51s.
• GitHub models are easy to use, requiring only a GitHub account to access and interact with models like GPT 40, which can be tested directly in a playground or through an API 15m59s.
• There are two ways to interact with the API: using the vendor SDK (e.g., Open AI SDK) or the Aure AI Inference SDK, with the latter recommended for building generic applications that can use different models 16m50s.
• The Aure AI Inference SDK requires only a GitHub token for login and allows for easy switching between models by changing an environment variable 17m47s.
• This makes it simple to test and compare different models, such as GPT 40 mini or Co:here, without requiring significant changes to the code 18m8s.
• Previously, integrating different models into an application would require writing specific code for each model, but the Aure AI Inference SDK simplifies this process 18m41s.
• The ability to easily switch between models makes it more accessible to play with and compare different models to determine which one is best for a specific application or use case 19m19s.
• The Aure AI Inference SDK also allows for easy AB testing and integration of new models, such as GPT 5, by simply changing an environment variable 20m4s.
• This feature is particularly useful for testing and comparing different models, as new models are being released frequently 20m10s.

Using GenKit with GitHub Models

• Genen kit is recommended for use with GitHub models due to its numerous plugins and integrations that make the process easier 20m48s.
• To use the Gen kit GitHub models plugin, one needs to install the Gen kit CLI and set up a Fireb Gen kit plugin, with instructions available in the Gen kit documentation 21m40s.
• After setting up the plugin, users can install the Jenex GitHub plugin in their JavaScript or TypeScript project to use GitHub models 22m14s.
• Genen kit works by configuring the framework and specifying the plugins to be used, with the GitHub plugin initialized by setting the GitHub token environment variable or specifying a line of code 22m37s.
• To use a GitHub model in Genen kit, users can call the generate function with a prompt and receive a response, as demonstrated in a simple example application 23m6s.
• The Gen kit GitHub models plugin allows for easy interaction with GitHub models, including the use of the generate function, tools, and images as input 24m59s.
• The plugin's capabilities include accepting structor input and output, enabling users to call large language models (LLMs) and retrieve information from them 24m36s.

Example Use Cases and Capabilities

• Example use cases for the plugin include translation, such as translating an image from English to Spanish by specifying the input and calling the GitHub model 25m30s.
• The GitHub model supports JSON output, structor output, and models for retrieval and generation in Genen kit, which is also available for creating embeddings 25m43s.

Awesome Fireb Kit Repository

• There is a repository called "Awesome Fireb Kit" that contains a list of relevant information about Firebase, including plugins for Fire Genen kit, which are official, supported by Google, and created by the community 26m31s.
• The Awesome Fireb Kit repository includes various plugins such as model embedding plugins, vector store plugins, monitoring plugins, and deployment plugins for Firebase functions 27m14s.
• The repository also includes plugins for the JavaScript community, such as Mine the Gen kit GitHub, Anthropy, Coher, Gro Mistral, Open AI, and Aure Open AI 27m52s.
• There are also plugins for vector store databases, including Convex Engine, NSW Quadrant, Astra PG, Vector PB, and Redis 28m5s.
• The repository contains evaluator plugins and other plugins, as well as official plugins for Go, such as Verx AI and AMA 28m26s.
• The AMA plugin allows users to run AMA on their laptop and use models like Gemma and Gemma CH billion without consuming tokens in the cloud 28m47s.
• The Awesome Fireb Kit repository includes talks, videos, articles, tutorials, and contributions, which are open to the community 29m14s.
• The repository is hosted on GitHub under the username "shabob" and is open for contributions and additions 29m35s.
• The repository is a crowdsourced list of plugins and resources, and users are encouraged to contribute and add new plugins and resources 30m1s.
• The repository emphasizes that Firebase is not the only option, and users can deploy their code wherever they want 30m24s.

GenKit and Goan Community

• GenKit is a framework for Firebase serverless functions that is simpler and faster to iterate than other frameworks like LineChain or Lama Index, making it easier to start experimenting with models 30m48s.
• The Goan community is happy about GenKit's support for Goan, a language that offers good performance and flexibility 31m21s.

NLU and Jenit NLU Project

• Natural Language Understanding (NLU) is a technology that identifies users' actions, such as understanding that a user wants to buy shoes from a specific brand like Nike 32m46s.
• NLU is used in chatbots to detect what users want and plan accordingly, and it is also used in Large Language Models (LLMs) 33m28s.
• A project called Jenit NLU was created using GitHub models and GenKit, which is a serverless project that can be easily deployed to Firebase 34m3s.
• The project has two parts: a Firebase function project and a function itself, which contains the source code in the index.js file 34m33s.
• To use GenKit, it is necessary to call the configure function to enable or add plugins, such as the GitHub models and the dot prompt engine 35m1s.
• The dot prompt engine is used to manage prompts, and it is one of the features that makes GenKit stand out 35m20s.
• OnFlow is a concept in Jenkit that allows for complex interactions by calling one flow from another, enabling the creation of more intricate processes 35m33s.
• To create a flow, a name must be given, and the input and output schemas must be specified, with the input schema defining the structure of the data being passed into the flow 36m5s.
• In this case, the input schema is an object type with a property called "text" that is a string, and the output schema is just a string 36m28s.

NLU Model and Intents

• The flow's main function is to define a schema, and before diving into the code, it's essential to understand the NLU (Natural Language Understanding) model 37m1s.
• The NLU model has intents and entities, which are used to extract or detect what users are trying to communicate 37m9s.
• In this example, there is an intent called "order shoes" with different utterances, such as "I want a pair of shoes from S brand" 37m28s.
• The entity "S brand" is defined in the entities JAML file, which can include multiple examples, such as "Puma", "Nike", and "Adidas" 37m59s.
• One of the benefits of using GitHub models and LLMs (Large Language Models) is that they can abstract and understand variations of examples, even if they are not explicitly listed 38m51s.
• The goal is to extract the ID of the intent and the value of the "S brand" entity, which can be sent to the backend 39m47s.

Output Schema and Code Implementation

• To achieve this, the output schema must be defined, which includes the intent and the list of recognized entities 40m21s.
• The code defines a list of shoes detected, which is used as an example, and calls a prompt named "nlu" that is stored under the "prompts" folder, with the actual prompt files being separate from the TypeScript code for abstraction purposes 40m48s.
• The "nlu" prompt is defined with a specific structure, including the model used (GPT-40), input schema (intents, entities, and user input), and output schema (NLU output in JSON format) 41m45s.
• The input schema includes a database of intents and entities, which are defined in JAML files, and the user's request, while the output schema is defined as NLU output 41m50s.
• The prompt is designed to detect intents and extract entities from a given text, with placeholders for intents, entities, and user input 42m39s.
• The code reads the JAML databases for intents and entities, which can be stored externally using Firebase or another storage solution, and calls the "generate" function with the intents, entities, and user request as inputs 43m18s.
• The result is an output in JSON format, which can be used for further processing 43m59s.

Genie Kit Developer UI and GitHub Models

• The Genie Kit has a developer UI that allows for testing and interaction with the models, including the ability to run the Firebase emulator and the Genie Kit UI locally 44m13s.
• The Genie Kit supports 23 GitHub models, which can be interacted with from the developer UI, allowing for testing and experimentation with different models 45m28s.
• Firest Jenit is a developer UI that allows users to interact with GitHub models in one place, eliminating the need to go to the GitHub Marketplace and choose a model separately 45m46s.
• The UI can be used to test models, such as the GPT 4 model, by providing input and receiving output, which includes intent, order, and detected entities 46m4s.
• The input schema is an object with a property called "text" that is a string, allowing users to type whatever they want and click on "run" to get the output 46m43s.
• The model can abstract and understand input in different languages, such as Spanish, and provide the same output as if the input was in English 47m44s.
• The model can also handle entities that are not in the database, such as a different brand name, and provide the correct output 47m11s.
• The example shown is a basic chatbot for an e-commerce site, but the possibilities are endless, and the lines of code required are less than 50 49m15s.
• The combination of Firest Jenit and GitHub models makes it easy to create apps, and the limit is only in the user's imagination 49m56s.

Firest Jenit Use Cases and Open Source Code

• There are already use cases for Firest Jenit, including Enterprise applications, despite the rate limits being low due to the beta status of the Giam models 50m41s.
• The code showcased is open source and can be found on the GitHub account "shabob" under the repository "jeny" and "shabobfarestYenkit", where users can access the source code, understand the usage, and explore the database of intents and entities 51m0s.
• The GitHub repository is open to contributions, and users are encouraged to fork, contribute, or open issues to help improve the project 51m40s.
• The creator can be found on GitHub at "shabop" and on Twitter at "savid", where they are active and open to feedback and discussions 52m10s.

Project Contributions and Future Enhancements

• The current GitHub models do not support audio or image generation, but the plugin only supports image and text input, and users are encouraged to contribute to adding audio support in the future 53m5s.
• The creator is open to feedback and encourages users to open issues or pull requests to contribute to the project, with a focus on adding new features and improving the plugin 53m52s.

Project Popularity and GitHub Universe

• The plugin has gained popularity, with around 128 downloads per week, and the creator is excited for the future of the project and the upcoming GitHub Universe event 54m16s.
• The creator will be attending the GitHub Universe event in San Francisco and will be hosting a session in Spanish, encouraging Spanish speakers to participate and ask questions 55m4s.
• The GitHub Universe event will be celebrating its 10th year anniversary, and the creator is excited for the new features and announcements that will be made during the event 55m45s.
• The speaker is open to invitations and will be giving a workshop in Glasgow at the end of next month, and also plans to visit Spain, specifically Madrid and Valencia, and is open to other invitations 56m32s.

Security and Contributing Guidelines

• For security purposes, the recommendation is to use tools like SonarCloud to check for vulnerabilities in every commit and set up a quality gate for every pull request 57m40s.
• GitHub has a dependency bot that can be enabled to check for security vulnerabilities in dependencies, and this feature is totally free 58m2s.
• To contribute to the project, one should go to the repository, read the contributing guidelines, and look for impactful contributions, such as creating real applications using the project's features 58m43s.
• The most valuable contributions would be to create real-world applications using the project's features, which would be fulfilling to see the project being used in real-life scenarios 59m1s.
• The project can be integrated into various web apps, including e-commerce, and there are many use cases for the project, but the speaker is looking for real-world applications using the project's features 59m38s.

GitHub Models Use Cases

• GitHub models can be integrated with a chatbot to provide a knowledge base for businesses, allowing users to ask questions and receive answers, and can also be used to summarize content, such as books or articles, by providing a brief summary of the main points 1h0m0s.
• Another use case for GitHub models is in e-learning, where a web app can be built to create an entire course with chapters, challenges, and quizzes based on user input, making it easier for creators and teachers to produce content 1h0m46s.
• GitHub models can also be used to create personalized quizzes for learning, which can help retain more information by constantly stopping and quizzing, and can be applied to various subjects and topics 1h2m13s.
• Additionally, GitHub models can be used for image-to-text applications, such as object detection, and can be integrated with other tools and technologies to build more complex applications 1h2m41s.
• A specific example of an app built using GitHub models is a baby food analyzer, which can detect the ingredients in a picture of a meal and provide suggestions for preparation based on the baby's age and allergies 1h2m52s.
• The baby food analyzer app uses a large language model (LLM) to detect the ingredients and provide suggestions, but it is essential to note that the app should not be relied upon solely for making decisions about a baby's diet 1h3m32s.
• A potential app idea using LLMs is a baby food advisor that detects ingredients like chocolate or sugar and provides warnings about their suitability for early babies, utilizing models like GBT-40 or ITAM model 1h4m17s.

GitHub Universe and Real-World Problem Solving

• There's a meme about using AI to solve real problems that take time and effort, rather than just creating art or writing books, which is a cool concept 1h4m52s.
• GitHub Universe is an event that offers a great experience, featuring talks, keynotes, new releases, and learning opportunities from real use cases and GitHub experts 1h5m58s.
• The event also provides excellent networking opportunities, allowing attendees to meet maintainers, open-source contributors, and important figures from the open-source community 1h6m24s.
• GitHub Universe offers various activities, including a lunch for certifications, a GitHub store with merchandise, and a chance to get a GitHub certificate by taking an exam for free 1h6m57s.
• The event is a great place to interact with people, including the creators of popular projects, who are often lovely and accessible 1h7m54s.
• The "hallway track" at GitHub Universe allows attendees to stop and talk to people they admire, creating opportunities for meaningful connections and relationships 1h7m47s.
• The event's lineup of speakers is epic, and the overall experience is amazing, making it an event worth attending 1h7m41s.
• GitHub Universe is celebrating its 10th year anniversary and attendees are encouraged to say hello and provide feedback to the team, with the event being for the attendees' benefit 1h8m16s.

Hosting Options and Giveaway Winner

• The difference between hosting with Firebase and GitHub Pages depends on the use case, and it is recommended to test both options to determine which one works best 1h9m21s.
• The speaker emphasizes the importance of using the framework that makes one work the happiest, fastest, and most productive 1h9m55s.
• A giveaway was held for a ticket to GitHub Universe, and the winner, Brian Shand, will receive an email with details on how to register and is invited to meet and be interviewed on the podcast 1h11m20s.
• The winner will also get to attend GitHub Universe in person and have a donut with the speaker 1h10m55s.
• The speaker thanks everyone for participating in the giveaway and congratulates the winner 1h11m10s.

Upcoming Events and Announcements

• An upcoming event called Open Source Bees will be held next Friday, where Shabi will demo Jit and his plugin in Spanish, catering to Spanish-speaking communities 1h12m7s.
• The speaker has one more announcement to share, which is about an event in Johannesburg, South Africa, where GitHub is hosting a community event 1h12m45s.
• The event in Johannesburg will feature a keynote by GitHub's CEO, a panel discussion, and participation from local partners, including Allin for Open Source Africa and Open Source Community Africa 1h13m30s.
• The event will also include a hackathon and will be an intimate gathering, with registration available on GitHub Constellation's website 1h14m5s.
• The event is scheduled to take place on November 14, and attendees are encouraged to sign up early, with the possibility of meeting the speaker and other community members in person 1h14m7s.

Closing Remarks and Johannesburg Event

• The speaker thanks the audience for their support of open source and congratulates the winner of an unspecified competition, with plans to send a message shortly 1h14m35s.
• The speaker invites viewers to leave questions and comments, which will be forwarded to Shabi for response, and reminds everyone to save the date for the Johannesburg event 1h14m47s.
• The event promises to be an epic day with swag, awesome sessions, and possibly donuts, with the speaker expressing excitement about meeting the community in person 1h15m0s.

Conference Experience and Atmosphere

• The conference has a super welcoming atmosphere and a community feel, which is amazing and makes the speaker feel grateful for the opportunity to speak there 1h16m7s.
• The speaker delivered their session and is looking forward to exploring the new sessions and learning from them 1h16m24s.
• The speaker is impressed by the coaching and organization of the conference 1h16m34s.
• The speaker has spoken at many conferences, but this one is different due to its non-corporate driven and smaller size, making it feel less loud and more approachable 1h16m43s.
• The speaker has made friends with many people at the conference and feels approachable themselves, enjoying the experience so far 1h16m50s.