An examination of ChatGPT and Dall-E’s capacity for UX.
Predictions that robots would replace human workers have been around since the 1920s [Karel Čapek. 1920. Rossum’s Universal Robots]. In 1965, Herbert Simon, recipient of the Nobel Prize of Economics and the Turing Award, predicted that “machines will be capable, within 20 years, of doing any work a man can do.” Certainly, machines already replace predictable physical work previously done by human hands, such as welding and soldering on assembly lines, food preparation, and packaging goods [Michael Chui, James Manyika, and Mehdi Miremadi. 2016. Where machines could replace humans — and where they can’t (yet). McKinsey Quarterly].
ChatGPT launched November 30th, and quickly exploded in popularity with over a million new users in it’s first week [ChatGPT Hits One Million Users, announced CEO of OpenAI Sam Altman. 2022. All Tech Magazine]. ChatGPT and its visual counterpart Dall-E has resurfaced the question; when will artificial intelligence replace human workers?
When will artificial intelligence replace human workers?
For the unfamiliar, ChatGPT is an AI-powered chatbot and a significant improvement on GPT-3 based chatbots you’ve likely encountered and become frustrated with. If you traveled over the holidays in the first year of the pandemic and experienced delays or flight cancellations, you may have used a GPT-3 powered chatbot on Delta’s app.
ChatGPT can do much more than respond to common help queries; it can write stories in Shakespearean style, translate English to other languages, and even write code. If ChatGPT can do a complex task like coding, what else is chatGPT and more broadly AI capable of? Does AI have the potential to replace knowledge workers like customer service representatives, front-end engineers, and content creators? What about user experience designers?
Despite predictions, artificial intelligence will not replace knowledge workers, and it certainly will not replace user experience designers any time soon. I’ll explain why.
Artificial intelligence is a misleading term. ‘Intelligence’ suggests a certain amount of creativity and novel thinking, qualities required by knowledge workers.
Artificial intelligence is a misleading term. ‘Intelligence’ suggests a certain amount of creativity and novel thinking, qualities required by knowledge workers. This ability to reason, generate, transform, and manipulate different types of novel information in real time is known as ‘fluid intelligence’ [Fluid intelligence. Science direct]. In contrast, ‘crystalized intelligence’ is “knowledge gained from prior learning of facts, skills, and experiences” [Cami Rosso. 2021. Can AI Enable Machines with Fluid Intelligence? Psychology today]. ChatGPT and Dall-E are capable of crystalized intelligence.
ChatGPT and Dall-E are capable of crystalized intelligence, knowledge gained from prior learning of facts, skills, and experiences.
ChatGPT is trained in part¹ using a ‘next-token-prediction’ technique whereby the model predicts the correct word based on the string of words that precedes it. For example, it might predict the word “mat”, “chair”, or “floor” if the sentence starts with “The cat sat on the”. The model is not conscious — it simply is using an algorithm trained to predict the next probable word [Marco Ramponi. 2022. How chat GPT actually works. Assembly AI].
Furthermore, ChatGPT and Dall-E (the visual equivalent of chatGPT) rely on a database to create an image or string of words. These databases reference existing language or images; they are not new or novel. Therefore, Dall-E will not invent a completely new style of art. Dall-E is most successful when you provide a noun and suggest a styling based on an existing artistic technique. For example, the query “astronaut in a flying hamburger, digital art” yields the following image (Figure 1). The same subject can also be created in the painting style of Monet (Figure 2) or Matisse (Figure 3).
Figure 1: Astronaut in a flying hamburger, digital art
Figure 2: Astronaut in a flying hamburger, Monet
Figure 3: Astronaut in a flying hamburger, Matisse
Dall-E is impressive because it generates images very quickly. Dall-E raises concerns on copyright and eurocentrism [Neel Dhanesha. 2022. AI art looks way too European.Vox], but I do not perceive it or similar AI technologies to be a threat to today’s human knowledge workers. Dall-E creates images based on an existing data set and is not capable of generating a new style of art. Therefore, Dall-E lacks a core tenet of human intelligence; ingenuity and creativity.
Dall-E lacks a core tenet of human intelligence; ingenuity and creativity.
It’s worth noting that MIT and Australian researchers are working to create AI capable of fluid intelligence. So whilst Dall-E and ChatGPT will not replace user experience designers today, machines capable of fluid intelligence may augment our work in the future.
To further prove my point, I tested Dall-E’s user experience design skills. I gave Dall-E common tasks, like creating a customer journey map, user flows, wireframes, and high-fidelity screens. What I learned is that Dall-E is good at roughly approximating the design artifacts produced by UX Designers, just as it’s good at roughly approximating certain artistic styles, but it lacks the project context, empathy, and creative problem-solving skills required to provide an insightful response.
UX Design is much more than the sum of our visual work; we guide product strategy with a deep understanding of our customers needs, business goals, and technical feasibility. That said, I wanted to test Dall-E’s ability to produce common deliverables such as a customer journey map, user flows, wireframes, and high-fidelity screens.
The following documents my queries and compares the work of a human user experience designer to Dall-E’s reply.
A customer journey map is a visual representation of synthesized research and is often the first artifact a user experience designer creates. Customer journey maps examine someone’s experience over time and help designers understand key pain points. These pain points can be used to inform design opportunities.
Dall-E’s response
First off, the text is illegible. The visuals here are arbitrary and look more like slides in a presentation. Being very lenient, the first and last images use similar visual elements to Adaptive Paths customer journey maps, but they are still clear misses. The results do not communicate a core journey, nearly all are missing time as the x-axis, and none of them highlight customer touch points or sentiments. They are unusable as customer journey maps.
I refined my search and asked it to create a customer journey map in the style of Adaptive Path, a leader in customer journey mapping. The results are still a nonsensical blend of personas, journey maps, and ecosystem maps.
Human designer
As a point of comparison, here is a customer journey map by Adaptive Path. Adaptive Path maps a person’s experience using the rail in Europe, mapping out key touch points, pain points, and emotions over time [Adaptive Path. Adaptive Path’s Guide to Experience Mapping].
“A user flow is a visual representation, either written out or made digitally, of the many avenues that can be taken when using an app or website” [User Flows. Interaction Design Foundation]. User flows can be exhaustive or represent a hero journey. They can be visually represented with squares, circles, and diamonds or a sequence of low fidelity wireframes. Designing at this level of fidelity helps designers quickly align on the key flow they are solving with stakeholders.
Dall-E’s response
Dall-E’s responded again with gibberish and a random assortment of images that look nothing like a user flow.
Giving Dall-E the benefit of the doubt, I specified my query and asked it to create a hero journey. The results were even farther away from what a real user experience designer would create, this time featuring a bad drawing of a hero. Comical and potentially career-ending if done in real life.
Human designer
As a point of comparison, here are examples of user flows from a real user experience designer [Camren Brown. 2021. What are user flows in User Experience (UX) Design? Career Foundry].
Once the designer has aligned on a key flow, the page structure, layout, information architecture, functionality, and intended behaviors can be fleshed out in wireframes. Visually, wires tend to look like architectural blueprints and communicate a product concept. Styling, color, graphics, and text is kept to a minimum [Jaye Hannah. 2022. What exactly is wireframing? A comprehensive guide. Career Foundry].
Dall-E’s response
If you squint your eyes, these could be wires: visually they look like blue prints. But if you look closer, nothing is communicated. When strung together, wireframes should communicate a product concept.
Human designer
As a point of reference, these designs are done by a human on Visme. It’s clear I’m looking at a financial services dashboard that allows me to view account balances over time and compare balances across currencies. The core tasks of the product are clear.
Finally, designers create a pixel-perfect design. The goal of high-fidelity designs is to refine the copy, spacing, color and visual elements used on the page.
Dall-E
Compared to the other prompts, this is Dall-E’s most accurate response. The page layout uses the rule of thirds, commonly used navigational elements, and visual hierarchy. However, the results are ubiquitous and notional at best. As a designer, I would not use these as a point of inspiration — I’ve seen these before — and certainly would not use them to replace my design deliverables. The color choices are poor, the page layout is outdated, and even the alignment of certain visual elements is off.
Human designer
This is the home page of Beans Agency, an agency out of Ukraine. Their home page is novel; they don’t use a common navigational system — users must scroll to get more information — and the imagery is unique. Although the text is light, it’s aligned with other visual elements on the page. Pixel perfect.
ChatGPT and Dall-E are impressive technologies; they both closely mimic human produced language and visual art. However, like other natural language processing models, they are limited by their training data. Eamon Majumder says “If the model has not been trained on a diverse and representative dataset, it may not be able to generate accurate responses to inputs that are outside of its training data.” [Eemon Majumder. 2022. ChatGPT-what is it and how does it work exactly?]
“If the model has not been trained on a diverse and representative dataset, it may not be able to generate accurate responses to inputs that are outside of its training data.”
For example, Dall-E raises concerns on eurocentrism as the results look like western art, suggesting it was trained primarily with a database of European imagery [Neel Dhanesha. 2022. AI art looks way too European.Vox].
I have no doubt that if Dall-E was trained with customer journey maps, user flows, wireframes, and pixel-perfect designs from design leaders like IDEO, Adaptive Path, and Frog, the results would look closer to work created by design professionals. Even if (and perhaps when) AI gets better at creating realistic design artifacts, AI lacks project context, critical thinking skills, and empathy that human user experience designers require to be successful.
AI lacks project context, critical thinking skills, and empathy that human user experience designers require to be successful.
Most importantly, today’s AI lacks creativity; it’s based on ‘crystalized intelligence’ and can not come up with brand-new ideas. While AI may become a tool to help our design process [Steve Dennis. 2022. How designers can start using AI at work today], it will not replace designers any time soon. After all, “The human brain is more powerful than any intelligent machine in existence” [Why our brain is the most intelligent machine of all. 2019. The University of Queensland, Australia].
¹ChatGPT also uses Reinforcement Learning from Human Feedback to reinforce responses from the next-token-prediction model. [2022. ChatGPT: Optimizing Language Models for Dialogue. Open AI.]
Customer Journey Maps. Interaction Design Foundation.
Edward Chechique. 2022. Exploring the Power of OpenAIChatGPT for product designers. UX Planet.
Anil Seth. 2022. Conscious Machines May Never Be Possible. Wired.
Kyle Wiggers. 2022. What to expect from AI in 2023. Tech Crunch.
Kelsey O’Connor. 2022. Is ChatGPT the beginning of the end for search engines?
