OpenAI integrates the Images in ChatGPT function, allowing users to create images directly in the chatbot through the power of the GPT-4o model.
On March 26, OpenAI said that the initial release of Images in ChatGPT mainly focuses on image creation and can be used by users of Plus, Pro, Team or free subscription packages.
On social networks, many people tried it out and expressed surprise about the new tool.
The Verge Citing OpenAI spokesperson Taya Christianson, the free version will have limited features, but will still outperform Dall-E.
“The new feature is a huge step forward compared to the previous model,” said lead researcher Gabriel Goh, adding that his team used the multimodal platform GPT-4o, one of OpenAI's most powerful language models, for ChatGPT's imaging capabilities.
According to Goh, a notable improvement in ChatGPT's image generation capabilities using GPT-4o is called "Binding" - a term that refers to the degree to which the AI image generator maintains precise associations between properties and objects. For example, given the prompt of a blue star plus a red triangle, a pattern with poor alignment produces only the red star without the triangle. Goh says most image models struggle with this, often mixing up colors and shapes if receiving multiple requests at once.
“The new visualization engine with Binding can accurately bind attributes for 15-20 objects without confusion, thereby demonstrating a significant improvement in accuracy and reliability,” Goh said.
The image generator on ChatGPT has also improved the display of text in images, helping to create text that is more coherent and not "distorted".
Additionally, the new tool uses an automatic regression method, which creates images sequentially from left to right and top to bottom similar to writing text, instead of the diffusion modeling technique used by most imagers.
“This is an iterative process that takes many months to perfect,” Goh emphasized. He added that while not perfect, the image creation capabilities on ChatGPT "have reached a point where the quality of the products created is usable."
In the new feature demo, OpenAI presents several examples that show ChatGPT's seamless imaging capabilities, such as a scientific diagram of a Newtonian prism experiment with precisely labeled color components;
However, compared to other models, Images in ChatGPT takes longer to create images.
“We will definitely improve the latency, but the existing ability to generate images, the quality of the images it produces can really make up for the extra seconds of waiting,” Shannon wrote on the blog.
Regarding the risk of creating fake, nude photos..., Shannon said Images in ChatGPT has strong protection features, preventing pornographic deepfake content and rejecting "fraudulent" requests, but did not mention details. The generated image also integrates C2PA standard metadata to mark it as created by AI, which can be looked up with tools for detection.
“Of course, no system is perfect, but we continually improve our protections,” Shannon added.
Bao Lam (according to The Verge, TechCrunch)
