Skip to main content

Artificial Intelligence

A Comprehensive Look at the Advances in Artificial Intelligence in 2023

 

Topic of the year:
Generative AI

It was obvious from the beginning of the year that 2023 would be a year with huge hype about generative AI. Because most of the tools were very easy to use, even people who were previously not interested in AI now started to use AI-based tools. Personally, I was surprised how many people started to use AI from the start. For example, in comparison to Instagram or Spotify, ChatGPT had their first 1 million users in less than a week.
GPT
 

When talking about Generative AI, it is useful to understand the concepts of Large Language Models and Diffusion Models.

 

Large Language Models

It's not possible to summarise 2023 without saying something about LLMs. The heart of all LLMs is transformer-based architecture. This type of architecture was designed in 2017, and one of the first LLMs was a model for generating embeddings, named BERT (Bidirectional Encoder Representations from Transformers). The power behind these models lies in their architecture, which allows us to process and learn from huge datasets and text sources. The models are trained on massive datasets that include a diverse range of text, which enables them to learn patterns, grammar and contextual relationships. The models have created a real revolution in generating human-like texts. The most common usage is the generation of text, summarization, translation or answering questions, which is something that can boost many businesses. 

When reviewing the topic of LLM, we also need to consider other aspects related to it, such as prompt-tuning or adaptation. To communicate with the model, users provide a prompt, which gives a command to the model. As more and more people started using ChatGPT, it turned out that some ways of writing prompts are better than others. Therefore, users started to structure the text in commands, which is well-known as prompt engineering. There are many courses and tips on how to get the best possible answers from ChatGPT by creating the best possible prompts. In some applications, one prompt is not enough, especially when an explanation is needed or when solving mathematical or logical tasks. It turns out that using a different strategy, a Chain of Thoughts, can be one of the solutions when carrying out such tasks. 

As the LLMs were trained on 'standard' forms of language, it was necessary to add more specialist knowledge in the case of certain specialised contexts, such as Law or Medicine. In small models, it's possible to carry out fine-tuning, but LLMs are huge or only accessible via APIs, which makes it impossible to create a new version of a model on, for example, laptops or local machines. One approach to fine-tuning is Retrieval Augmented Generation (RAG), where your prompt provides the source of the new knowledge for the model. Note that OpenAI does offer fine-tuning and has its own instance of the model, but you need to prepare a good quality dataset for training, and training on a big dataset is costly.

Another topic around LLMs which is worth mentioning is LoRA (Low-Rank Adaptation of Large Language Models). It is a training technique that significantly reduces the number of trainable parameters. It works by inserting a smaller number of new weights into the model and only these weights are trained. This makes training with LoRA much faster, more memory-efficient, and produces smaller model weights (a few hundred MBs), which are easier to store and share. 

Access our e-book

 
If you want to receive access to our e-book please provide your email address.

A Comprehensive Look at the Advances in
Artificial Intelligence in 2023

2023 finished almost a month ago and it’s high time to summarize what happened in AI. It was a very exciting year, where innovation and breakthroughs have shattered preconceived boundaries. We were witnessing how fast AI solutions can spread among users and how quickly they can change our lives. This article aims to be your compass through AI’s developments in 2023, distilling the essence of groundbreaking research, and transformative applications that have accompanied this technological revolution.

Digica Solutions would like to keep you up-to-date with information on Digica services. For information on how we will use your data, international transfers of data and your rights, please see Full Privacy Policy details.

Agree to terms and conditions

Please enter a valid e-mail address

Invalid Input