Based on materials from theverge.com
Modern artists, in theory, should start worrying about whether they will lose their work due to inexorable automation – including creative processes. However, what if you look at the situation from a different angle? What if artificial intelligence starts drawing, editing, and other types of work for the artist? AI tools are already being used to automate time-consuming processes that previously had to be done manually. And the results do not indicate a potential infringement of AI on the work of a human artist, but, on the contrary, the benefits for creative activity.
Companies that make creative tools that have become the industry standard, such as Adobe and Celsys, have added AI to their digital creativity software in recent years. They hope this will speed up the workflow by automating the workflow, giving artists more time to express themselves and experiment. AI integrates seamlessly into software, but it has a surprisingly big impact, from machine learning tools that allow you to find specific video frames faster to functions that allow you to color in outline drawings at the click of a button.
“The best AI features can help artists by freeing them from repetitive routines,” says Tatiana Medgia, who runs Sensei, Adobe's AI platform. Her judgment is based on a study by Pfeiffer Consulting, commissioned by Adobe, in which most of the creative professions said they are not afraid to be replaced by AI and that they see the main potential of AI and machine learning when applied to tedious, uncreative tasks. For example, this could mean smart photo cropping, which automatically recognizes the scene in the frame, or auto-tagging, which helps people find stock photos faster. 'Artist control is still needed. Creativity is fundamentally deeply human, says Medzhia. “AI cannot replace the creative spark.”
Every time Adobe reveals the concept of an AI-powered tool, it produces a wide resonance, such as Content-Aware Fill for video. But usually this is a more subtle use of AI. Recent additions include an automatic audio mixing feature in Premiere and the ability to create searchable PDF images using OCR in Adobe Scan. Yes, this is not as effective as artificial intelligence, which automatically removes unwanted elements from the video, but this is quite important, since it frees the creative process from some of the routine actions.
There are other AI-powered features in Adobe software that can have a significant impact on creative productivity. One of the functions, for example, takes a video of a dog jumping into a pool and generates tags to describe it. Another might take a sketch of a mushroom and pull up photos from the web that look the same – similar to Google's experimental Quick Draw and AutoDraw tools, which use neural networks to recognize sketches.
When watching a video of a dog jumping into a pool, the AI generates tags to describe it, such as an object and an action. A timeline below the video shows when the action starts and when the subject appears in the video so that the editors can find specific scenes more quickly.
Other AI tools can find more serious uses for the artist's work, such as the auto-coloring tool made for comics and animation. The beta version of Celsy's manga and illustration software Clip Studio now has an AI-powered feature that can automatically color a black-and-white outline image with little artist input. The results can be unpredictable and require a bit of polishing, but there is a lot of potential in the way the technology can be used by artists and entire studios.
“AI-powered coloring tools could play a big role in the future of 2D animation,” says animator Joao Do Lago, who has worked on several animes, including Netflix's Castlevania. Do Lago is confident that they can give the artist the freedom to experiment by not having to spend time painting every frame.
Users fill in the outline drawing with 'hints' where a particular color should be. The entered data is sent for processing to the Clip Studio servers (this part requires an Internet connection), and then “returned” back to the program in the form of a fully colored image.
“One of the things that makes animation really difficult to produce is the amount of time it takes to create, which forces most studios to stick to a style and formula that has already been tried out in previous work,” says Do Lago. “But when you can automate most of the process, you have more time for different ideas, learning different visual language options, because you can implement them much faster.”
A number of studios have already started investing in the study of automatic coloring, including OLM, the studio behind the Pokémon anime. AI-based coloring tools have been in use for several years now, such as PaintsChainer, a browser-based tool from Japanese startup Preferred Networks, which has already been used by a number of manga publishers, and Style2Paints, a network bot created by a group of students from China University of Hong Kong and Suzhou University.
Another test using the Colorize tech on one of my animations. This time with some manual tweeking to try to achieve more controled results. It's very limited tech, but the results are still impressive. No doubt A.I's will play a huge role on the future look of 2d animation. pic.twitter.com/e8mEdlI0Wg
– Joao? (@JonnyDoLake) December 18, 2018
Celsys says their AI learns through sets of contour drawings made from color illustrations. The technology is based on deep learning, combining computer vision tools such as those found in self-driving cars with visual content creation systems such as those used by Nvidia to create highly realistic AI-generated faces. It is important to note that the company also uses the data uploaded by the artists to improve the technology, although Celsys says the artists will retain the copyright of the uploaded and generated images and the data will never be released.
Celsys is optimistic about how the tools they create will serve the interests of the artists rather than trying to replace them. “We think AI-powered features are just a type of tool within the scope of digital art,” says a Clip Studio spokesperson. “We hope creative people use these tools to the best of their ability.”
What do you, dear readers, think about this? Can you consider a real work of art that in the creation of which artificial intelligence helped the artist? Where is the borderline between art and soulless stamping for you?