Joan Holloway

Beginning 2023 I joined a team to compete in a hackathon about AI organized by Lablab.ai.

Our goal was to improve the organization and access to multimedia information in an enterprise context using AI.

For this we developed a web application that transcribes videos and organize it by chapters so its easier to navigate, then its content in put into a Markdown document so the user can fix potential mistakes by the AI or add aditional information.

After this document is saved its content is sent to an LLM to generate vectors which in turn are saved in a special database, this makes possible to query them using natural language (like asking a question) with the help of a semantic search engine.

The LLM powering this search engine is multilingual, this means users can query (or ask) in multiple languages (spanish for example) and our search engine will “understand” and return the relevant results for the question, both english and spanish ones.

When a document is saved a new page is generated within the site structure.

This page is designed so the user can consume its content both in its transcribed version (text) and in its original form (video) at the same time. This means the user can start reading the content and at some point just play the video from the current reading sentence, the other way around is also possible.

In this project I learn to work with AI, implement plugins and comprehend extensive code bases to extend its functionalities.

Also it challenged my english skills (I’m a spanish speaker) because my team mates were from different parts of Europe so our communication had to be in english.