In the ever-evolving landscape of technology, there are few innovations that capture the attention and curiosity of both technical and non-technical users.

One such breakthrough is ChatGPT, a viral piece of technology that has allowed people to experience the power of Artificial Intelligence through meaningful conversations. At Software Mansion, we were inspired by the potential of this technology and wanted to explore how far we could push its capabilities in a practical application. This led us to develop Membrane Bot, a revolutionary project aimed at transforming technical support by leveraging the capabilities of AI and conversational agents.

As the creators of Membrane, a high-level multimedia framework written in Elixir, we understand the value and complexity of its ecosystem. With over 100 plugins and the core framework to maintain, finding specific plugins tailored to individual use cases became a daunting task. We realized the need for a solution that could simplify this process and provide users with personalized recommendations effortlessly. That’s when the idea for the Membrane Bot, an AI technical support bot, was born.

We knew that fine-tuning an existing Open Source Large Language Model (LLM) has the potential to be a solution to our problem. However, we recognized several significant drawbacks to this approach. Firstly, the cost and effort of ongoing updates required to keep up with the ever-evolving Membrane framework threatened the feasibility of this strategy — the last thing we want is to spread the team even thinner with additional steps. Secondly, the lack of suitable base models and potential licensing issues with existing models limited our options. Additionally, fine-tuning an LLM requires a lot of computing power and is very expensive.

Taking a different route, we opted to integrate a semantic search database into our Membrane Bot, leveraging a framework called ChromaDB. This database encompassed a wealth of knowledge, including the Membrane Guide, Tutorials, and technical documentation of all Membrane plugins. By effectively processing and organizing this information, we aimed to create a conversational assistant that had a comprehensive understanding of the Membrane ecosystem.

In order to extract in-depth knowledge from the model, we developed a two-step querying system. First, by providing the GPT model with the conversation history and feeding it glossaries of multimedia terms, we encouraged the bot to think about multimedia processing, input, outputs, and transformations necessary to go from one to the other. This allowed it to break down complex multimedia-related problems and solve them step by step, fetching relevant information from the knowledge base. Our focus was on empowering the bot to provide assistance that went beyond the surface level.

Throughout the development process, we encountered both successes and challenges. On one hand, the Membrane Bot impressed us with its ability to answer complex questions and provide valuable guidance. However, we also noticed limitations, such as frequent mixing up of terms, eg. suggesting demuxing the MP4 with the muxer. Additionally, maintaining consistency in responses proved to be a challenge, even with attempts to disable randomness by adjusting the temperature to 0. It’s important to note that we achieved much better results only when using GPT-4.

Given that OpenAI limits the monthly usage of GPT-4 models and the potential of it providing misleading replies, this particular implementation will not be making it into production in the current state. That said, it will be available for a limited time as a demo for those curious readers.

Looking ahead, we believe that fine-tuning existing LLMs with multimedia-related content could be the key to overcoming these challenges. By training the model with domain-specific knowledge, we would hope to achieve more accurate and contextually-aware responses from the bot. Although cost, training data availability, and update ability issues currently restrict this approach, we remain optimistic about its potential in the future.

The Membrane Bot represents our commitment to pushing the boundaries of AI and enhancing technical support experiences. By leveraging conversational agents and AI technologies, we aim to revolutionize the way users interact with the Membrane Framework. The journey has been filled with valuable insights, successes, and challenges, revealing both the incredible potential and room for improvement in the field of conversational AI. As we continue to advance the Membrane Bot, we look forward to delivering a seamless and intuitive support experience for users, further strengthening our commitment to excellence in the world of technology.