Project development included the creation of a contextual search system that analyzed the statement of work and provided answers to questions based on internal documentation.
The project is aimed at deploying a language model and creating an interface for English to Finnish translation, which helps the company’s employees. The model is trained on customer data, ensuring accurate and high-quality translations.
Challenges
-
Deploying the language model: One of the challenges was to successfully deploy a powerful language model capable of processing and analyzing texts at a high level.
-
Interface creation: We designed a user interface that provided a user-friendly and intuitive look and feel when using the English to Finnish translator.
-
Training the model on customer data: One of the key challenges was training the language model on data provided by the customer.
-
Performance and resource management: Processing texts using language models can be a resource-intensive task.
-
Translation quality: We strived to provide high quality translations from English into Finnish.
Solutions
-
Designing and building our own local ChatBot, similar to ChatGPT, based on the LLama2 architecture model with 13 billion parameters. We researched different language models and selected the most suitable one for our needs.
-
Interface creation: We designed the user interface using modern web technologies. The interface was intuitive and attractive, with text input for translation and instant results.
-
Training the model on customer data: We developed a process to train the language model on customer data. This involved pre-processing and cleaning the data, selecting a suitable model architecture, adapting hyperparameters, and repeatedly training the model to achieve high quality translations.
-
Performance and resource management: We optimized the performance of the model using various techniques such as batch word processing and parallelized computation.
What We Have
-
We successfully deployed a powerful language model on a local server, providing fast and reliable text processing. The model was ready to use and accessible through our user interface.
-
We successfully trained the language model on data provided by the customer. The model was customized to meet the specific requirements of the customer, achieving high accuracy and quality translations to meet their specific needs.
-
We efficiently managed the model’s performance and resources, ensuring fast text processing and minimizing time delays. The model was optimized for efficient use of computing power and resources.
-
We achieved high quality translations from English to Finnish. The translations obtained using our language model were accurate, grammatically correct and had a high degree of correspondence to the source text.