When you read the sentence- “a monkey reading a book on a boat” you may definitely create an image of it in your head. But can you imagine a software or system that will help you generate actual images like this? With technology and AI getting a boost you can certainly expect a system that will give you the ability to do this. But can you think of a system that can create random images from the scratch?
The new technology- DALL-E 2 by Open AI will blow your mind with the accuracy of images it can generate using plain text. DALL-E 2 comes after DALL-E, only with improved image quality and resolution of the output. The machine learning model reflects the milestone that the AI community has reached today. The new AI system uses text to create realistic images that have never existed before!
The CEO of OpenAI, Sam Altman, and some engineers of the company shared images that were created using the tool on Twitter during the announcement of DALL-E 2.
How does DALL-E 2 work?
The image above has been created using something that AI researchers call a neural network also known as a diffusion model. The neural network is a system that has been modeled similarly to the network of neurons in the human brain. This helps to analyze a large amount of data or photos by creating and looking for patterns.
DALL-E 2, through deep learning abilities, not only understands simple objects like books, motorcycles, etc. but also understands creating a link or relation between them. It uses the CLIP and diffusion model together to create stunning images.
The CLIP model is a type of neural network that has been trained on a variety of images and texts pairs, particularly on 400,000,000 pairs. When these two models are paired together, they create a brand new image from scratch that matches the description.
What can DALL-E 2 create for users?
DALL-E 2 is a generative model that will provide outputs using the description that is given by you. If you check out the DALL-E 2 blog post on Open AI’s website, you will come across an interactive format to give you a demo of how the system works along with its research paper. It also consists of an explanatory video that will showcase the creation of realistic images by combining concepts, attributes and styles.
Apart from creating new images through text, DALL-E 2 can also retouch and edit existing images by replacing what you don’t want seamlessly using AI-generated imagery. It is also capable of creating multiple images of the same variation using different variations and styles.
The tool helps people to express themselves better visually in ways that were not previously available. The tool also works vice versa by helping humans understand how the world of AI works.
As explained in the video, DALL-E 2 is like a person who has learnt words or objects using labels or images so it will provide you the best possible output by creating a relation between objects or even actions. The limitation arises when the tool has learnt something incorrectly then it provides an inaccurate output based on what it has learnt. AI researchers can overcome or refine the limitations by feeding large amounts of data.
Although people have highly appreciated and are excited about the wonders this tool can do but there are experts who are concerned about the use case of this tool. It can be used to create beautiful art pieces and even something out of this world but the flip side is that it can also be used to create that have the potential to cause destruction. The tool can be used to create fake and misleading photos too.
Another aspect that causes worry is whether the tool will help artists or it will hurt them. The advancement in tech has always resulted in the replacement of humans with machines but does this tool have the potential to do that? Or will it aid artists and illustrators in creating more aesthetic work?
This system is not allowed to be used by everyone yet and is only available to people who have signed up for the waitlist. The company plans to release it to everyone eventually after testing its limitations, risks and capabilities.
On the other hand, experts also believe that development of such systems will ultimately help company improve their search engines, digital assistants and other technologies. It will also help programmers, graphic artists and professionals from other fields automate several tasks.
Featured Image Credits: Open AI/ DALL-E 2