Free guide: 5 ways to test and measure UX content. Download now!

UX CONTENT RESOURCES

How is UX writing impacted by AI?

Why skills will always come before tools

Like every industry, UX writers and content designers were taken by surprise when generative AI tools were made widely public in 2022. What’s even more surprising is just how useful the technology has been for writers in design teams.

As it turns out, AI tools have been a fantastic addition to day-to-day writing work. In March 2024, we conducted a survey of more than 150 working content designers, and found that 82% use Large Language Models in their day-to-day work. 

Over 50% of UX writers and content designers find LLMs either “very” or “somewhat” useful.

Although we’ve been writing about generative AI for years, no one could have predicted just how quickly Large Language Models have become a daily part of work. At first, it seemed as though AI might have a shrinking effect on the market, but that hasn’t been the case. 

But there’s a reason why content designers and UX writers find AI useful: they already have a substantial amount of knowledge and skill to make AI work for them. Without that knowledge, LLMs are just a toy. It’s like handing someone a circular saw if they have no carpentry experience. What would be the point?

So when it comes to the question, “can AI replace UX writing?,” the answer is more complex than just “yes” or “no.” It really depends on the skill of the individual. 

And we’re quickly discovering that there is a gap between UX writers and content designers who know how to embrace AI, and those who are being left behind.

In this post we cover:

How is AI changing UX writing and content design?

It’s important to remember that any new technology brings a certain amount of hype that isn’t fully supported by the technology itself. Our imaginations run wild and we picture all types of drastic scenarios.

Immediately after OpenAI released ChatGPT in 2022, it was speculated jobs would soon disappear replaced by digital assistants. The truth is much more nuanced. Although the tech market did suffer an adjacent decline due to over-hiring, many jobs have returned—and unemployment in the United States remains low as of April 2024. Overall, AI hasn’t caused a surge in job losses.

But that doesn’t mean AI hasn’t had an impact. As we discovered in our survey of more than 150 UX writers and content designers, AI has had a substantial effect on people who write user interface text. Many of those people now rely on AI as a day-to-day design tool, much like Figma, Miro, or Notion.

That efficiency may lead to a consolidation of teams, but it hasn’t so far. In fact, the 2024 jobs market for content designers and UX writers remains strong. 

But there’s a reason why these content designers and UX writers find AI useful. They already have the skills and knowledge to do the job well—and therefore know what to ask an AI assistant as part of a work process. That knowledge is half the reason why AI is useful in the first place.

“I consider AI as an assistant to designers. Usually, designers work alone and share their work later for feedback. But with AI, I feel like I can ask questions, share copy, and improve my guidelines. In general, it helps me with decision-making.”
Fatiha Belabed
Content Designer, Mews
“I'm currently a Content Design team of one. I've come to depend on tools like chatGPT, Writer, and my company's internal LLM to help me create templated content, give me ideas, and be my sounding board. Now that I've played around with these tools for myself, I'm starting to explore ways to use AI to distribute voice and tone, terminology, and snippets of reusable strings for the designers, engineers, and product folks on my team who also create content.”
Sarabeth Blum
Senior Content Designer, Spotify

Why UX skills always come before tools, including AI

Let’s talk about why some content designers and UX writers find AI useful for design, and why some don’t. But in order to do that, we also need to understand how language models work.

Unfortunately, the discussion surrounding Large Language Models (LLMs) and content has been fairly shallow. It’s easy to think AI can do it all for us, and that creating content is as easy as crafting the right prompt. But that assumption is flawed, especially as it relates to content design.

Why? Because any type of content—whether it’s code, prose, or an image—relies on human eyes to evaluate it, and then place it within a context that makes sense. It’s how content designers understand the difference between a message that surprises a user and makes them happy, versus one that would offend them.

Thus, in a time where AI relies on user-created prompts, skills like UX writing fundamentals are more important to know than ever.

Without any knowledge of UX writing or content design best practices, using AI to create content is like painting with your eyes closed. You have the tools, but you have absolutely no control, direction, or context.

“Without any knowledge of UX writing or content design best practices, using AI to create content is like painting with your eyes closed.”

It ignores crucial questions like: which tasks? Why those tasks and not others? What part does human intervention play in any type of editing process?

These are important questions any design leader or professional needs to grapple with. If your hope or intention is to use AI to create text for user interfaces, then simply jumping into a prompt window isn’t going to help you. The act of creating UI text is only part of the full content design process. And it is, indeed, a process with several steps that require full attention.

If you stroll into an AI prompt window with absolutely no knowledge of UX writing or content design best practices, you are relying on the AI itself to guide you and tell you what to do. 

There are three critical aspects of any interaction with AI that define whether the output will be useful…or useless. 

#1. Prompt structure 

When you talk to an AI, whether it’s ChatGPT, Gemini, Claude, or a custom language model, you’re giving it a “prompt.” AI will take that prompt and analyze the various words in it to understand your meaning. Part of the way it will do that is by analyzing the relationships between the words you provide.

The more detail you provide in your prompt (and the better structured it is) the more likely you’ll receive a useful response. (We tested this out in a podcast episode.)

That means you need to spend time with assistants in determining how you should structure those prompts, what you need to include, and how long they should be. These change based on whatever tool you’re using and from one model release to the next. What worked today may not work next week, which requires constant fine-tuning.

Like Figma, UX writers and content designers should treat AI models as ongoing tools that require practice and comfort before becoming proficient. It isn’t enough to just treat them as chatting with a friend.

#2. Context 

An underrated element of using AI to create design experiences is context. AI isn’t all-knowing. It isn’t able to write strings or strategies for you based on nothing. It doesn’t just require a well-written prompt, it also requires context on your specific design problems—which it won’t be able to gather on its own.

Very few organizations are able to rely on user personas that apply to a broad population set. Even in very generic examples—like, say, a supermarket—there will be user segments with very specific needs. AI assistants don’t have any knowledge on those personas until it’s been taught to them. (This, by the way, is behind the rise of more assistants powered by a company’s proprietary knowledge and user documentation, rather than generic AI models.)

This means any prompt you provide AI needs to include critical information about your design phase, your users, and any relevant research. 

This brings us to our third point…

#3. Editing skills  

Editing is an underrated skill. Any AI is going to provide you with an immense amount of output— you may get 20 or 30 pieces of content to choose from for any particular design. (Only asking for one is a fool’s errand, which we’ll touch on shortly.) With that much content to choose from, how are you going to decide which to use?

The answer is simple: you need the skills and grounding in best practices to understand which suggestions from AI to adopt, and which ones to discard.

Consider a simple request, like creating an error message. We could simply ask an AI to create an error message for our particular context, but that’s only part of the problem. The next is determining whether the string matches best practices. For instance:

  • Does the error headline state a single question?
  • Does the CTA match the verb used in the headline?
  • Does the description describe any consequences?

If a message created by AI ignores these points, then it doesn’t matter that it was created faster. It ignores best practices, and is effectively useless. The user experience is still poor.

This is why, when it comes to artificial intelligence and design, learning proper skills and best practices should always come before using tools.

AI is an assistive technology, not a final one

Part of the problem with using AI as a tool to generate large amounts of text is that one can easily reduce the UX writing process to an output. Stating that UX writing is equal to an output is like attempting to judge a software engineer’s value by the number of lines of code they write. The output is not an accurate representation of the breadth of work that has gone into the project.

The truth is that by the time a UX writer or content designer gets to the point of writing strings, most of the work has already been done. Or, to put it more accurately, the work is an ongoing force that will require constant revision and attention. Strings and other types of UX content are not created in a vacuum, then left to serve a role with no review. Iteration is a process that any content designer will undertake.

How to use AI during the design process

As our survey shows, even very experienced content designers are able to use LLMs in their day-to-day work.

What’s important to remember, though, is that AI shouldn’t change the ways design is executed. It’s a tool, not a process. So how can content designers and UX writers implement it properly?

We like to follow the “Double Diamond” method of design, which features four distinct phases:

  • Discovery
  • Define
  • Design
  • Deliver
"There are opportunities at my job to experiment with AI, and content designers can get involved if they're proactive so I've found a couple opportunities to explore. As it's all so new, there aren't many rules, so everyone is exploring together."
Jessica Bowler
Senior Content Designer, Bumble

Step 1: Discovery

During this phase, the team is researching as much as they can about a specific topic or problem. This may be a broad question, or it might be something specific to a particular user base or segment. 

Some of the actions UX writers and content designers might conduct during this phase include research keywords, even speaking with users via surveys or interviews, analyzing competitors, and doing general research on a particular topic area.

Note how many of these actions aren’t actually creating any content yet. They’ll be in the service of discovering content itself. Much of the “gold” to be found won’t lie in generic descriptions of types of users, but in the specific user bases themselves.

For instance, a team working on a supermarket app doesn’t want to know about people who buy groceries in general. They might want to know about people who buy groceries who are very busy, or about people in a specific economic category. 

That information can’t be found using any one AI alone for several reasons, not the least of which being that AI doesn’t have a specific training set on certain customers. It has general information. It can also hallucinate, so providing general insights makes it hard to fact check any conclusions.

Instead, UX writers and content designers using AI during the discovery phase are more productive if they create tools, templates, or other material. 

For instance, teams can use AI to:

  • Create surveys to give to customers
  • Analyze metrics or product telemetry to identify patterns
  • Help create draft reports 
  • Synthesize relevant user comments or feedback 

Step 2: Define

During this phase, the design team begins to hone in on the problem statement. At this point, the UX writer or the content designer may engage in some early brainstorming and tests for content direction. It’s crucial that at this stage, the team has some type of insight to ground their direction.

But UX writers and content designers can’t just rely on brainstorms from generative AI tools. They need real feedback from users to understand what the direction should be.

Research like 5-second tests can be crucial here. Participants are given variants of text to examine for 5 seconds and are then asked a series of questions. Here, AI can be useful in creating variants for these types of tests (though keeping in mind, as we explained earlier, that the AI will need as much context as possible to create usable variants.)

Step 3: Design

During this stage, teams take the problem statement and start developing prototypes to solve it. This is where much of the traditional building and testing takes place, along with the bulk of the writing for a user interface. Though keep in mind, the first two phases have already set the groundwork for direction. 

This is where artificial intelligence can play a large part in helping a content designer scale their efforts. Tools like ChatGPT or Gemini can create multiple variants, allowing teams to pick and choose what works best.

The picking and choosing can only work if content designers and UX writers create fully formed prompts with context. That context needs to include:

  • Information on users
  • Any relevant research into spending patterns
  • Insights from 5-second tests about preferred terminology
  • Information on style and tones
  • Structure and format of content required

Any content designers or UX writer examining AI-created content during this phase needs to ensure it remains consistent with other parts of the product. Inserting the appropriate terminology and auditing content for consistency can take a significant amount of time depending on the product.

Step 4: Deliver

In this stage, the product is delivered and released. From here, content designers iterate and test new content, track performance, and send updates.
 
How will AI impact this area of design? Content designers can use AI to brainstorm ideas for further testing, or testing synthesis. AI may even be able to gather performance metrics and change content dynamically.
 
This raises even further questions! How would content designers even manage such a system of content? If content is being created dynamically, how can content designers and UX writers make sure that content stays within a set of boundaries or parameters?
 
That’s exactly what we’ll discuss next.

How does AI affect content strategy?

So far, we’ve focused solely on how to use AI within the context of writing individual strings—the stuff that most content designers or UX writers will do day-to-day. But as design professionals rise in seniority, more of their work is focused on strategic goals.

How does AI play a part in such work?

This is an existential question. The very nature of artificial intelligence means the assumptions about how content strategy could or should operate are now in flux. This doesn’t necessarily mean that principles about UX content within a product change, but it does mean how we go about achieving those goals may very well change.

Let’s start here. This is a model created by James Garrett in his book “The Elements of User Experience” to describe how a product is built. As you create foundational structures (strategy) you move higher up the scaffold to add more, creating further clarity. Content plays a significant role here. From a structural perspective, the existence of AI doesn’t change much if at all here. 

Each product still needs a foundational strategy, content requirements that are articulated clearly, along with an IA, navigation, and surface design that makes sense to users. (There is something to be said for the concept that AI can create individual interfaces, but that’s another topic!)

A diagram showing the elements of user experience across 5 levels: strategy, scope, structure, skeleton, and surface.

“The Elements of User Experience,” James Garrett

However, the closer we get to actually delivering content, the more we can start to see how AI can affect content strategy.

To look further into the concept of content strategy, let’s take a look at Kristina Halvorson’s definition: “We still define content strategy as guiding the creation, delivery, and governance of useful, usable content.” 

The Brain Traffic model explains how content strategy can really be broken into two parts: “content design” and “systems design.” Each contains two further sections that cover a different area of content.

A quadrant showing the four elements of content strategy. Editorial and experience fall under content design. Structure and process fall under systems design.

The Content Strategy Quadrant,” Kristina Halvorson 

Each of these areas carries significant questions. For instance, in the “editorial” section, we might ask questions like: what is our voice and tone? What is our style? For “experience,” we might need to ask what formats our in-product content takes. Similar questions follow for process and structure. For instance, how is content collected, distributed, and iterated upon?

Now, consider how the availability of AI tools changes how we approach these questions.

How does AI change editorial?

It’s important to point out that most of the initial questions we ask about editorial remain the same. For instance, style guides, tone of voice, localization standards, etc. These are all important to identify. What AI changes is how these are all implemented.

For instance, style and tone guidelines can be added to custom Large Language Models, which are then accessed by others to create new content. They essentially guide the creation of new content without the need for team members to manually check style. 

This isn’t just a practical question, it’s a strategic one: how do you create your editorial strategy in a way that assumes the involvement of these types of AI tools?

How does AI change experience design?

This is where the addition of AI starts to create significant changes in policy.

How content is housed, what format it takes, and how that content is distributed, might all change significantly due to AI. Assistants within design systems can create suggested text based on editorial guidelines, and may even pre-fill components with text. 

A design system within Figma, for instance, may at some point reference component-level guidelines so an LLM can create placeholder text. How these guidelines are distributed and accessed by AI are questions any content strategist should be grappling with.

For instance, is it possible that user interface strings could at some point by generated by an AI system that learns and grows based on that user’s behavior?

In the future, could tooltips be dynamic and rely on an array of strings, or even strings generated by a custom trained model? 

If so, how would content designers determine the principles and best practices those strings would follow? How would that impact content within a navigation? 

These are all questions that require a mind experienced in content and content delivery.

How does AI change structure?

This may be the category where the introduction of AI creates the most changes. More AI systems are improving methods of retrieval-augmented generation (RAG) which rely on documents with rich metadata and tagging systems.

More content designers are already finding themselves in a position of having to train custom models with existing content. If that content isn’t yet formatted in the right way, that role often falls on designers.

Consider the questions that Brain Traffic puts to content strategists about structure:

  • How will we organize content for browse-and-find?
  • What tags are most intuitive for users?
  • How will we categorize content for efficient management?
  • How will we structure our content for future reuse?
  • What are the requirements for personalization, dynamic delivery, AI?

Consider a product that uses a custom chatbot to produce dynamically created help content. In that instance, the chatbot would rely on a database of help content that is sufficiently tagged, organized, and can be retrieved. This is all work that needs to be done by content strategy.

How does AI change process?

Finally, AI means updating processes with a significant amount of tracking. Content strategists will need to record and track which documents are used to train custom models, when they’re changed, who changed them, and with what data. 

This is essentially version control for content powering AI models. 

The existence and even the possibility of future AI systems that create content all raise serious questions for content designers. The areas of content strategy outlined by Halvorson have not changed, but how we provide them to users has changed. 

As a content strategist, you need to think carefully about how AI would impact each of those areas—and therefore, how AI-informed content strategy would interact with other areas of a product as well.

How AI changes the delivery of in-product content

To expand on one of the topics covered by the content strategy quadrant, we’re already beginning to see an emphasis on how content is structured so AI can take advantage of it for in-product delivery.

One of the more obvious places this is happening is knowledge bases. An AI Knowledge Base is essentially a repository of information that users can access on demand. Instead of having to manually update this information, an AI-powered Knowledge Base is able to dynamically create content. 

As more users ask queries, give feedback, or just generally provide any type of information, the Knowledge Base then takes that data to create new types of content. This might take the form of snippets—short pieces of text—or it might even constitute entire help articles.

An example of this might be a product like Zendesk, which analyzes incoming support tickets to create new content. There are other examples, such as Capacity or Starmind.

At the heart of an AI Knowledge Base are some pretty sophisticated content practices that any strategist needs to be aware of. One of those is something called a knowledge graph. Now, the concept of a knowledge graph has existed for decades, but they become even more important when introducing AI tools.

IBM defines a knowledge graph as something that “represents a network of real-world entities—such as objects, events, situations or concepts—and illustrates the relationship between them.” 

You see the outcomes of knowledge graphs every day, even if you don’t know how they work. Every time you get a suggested purchase recommendation on a website, or get some content recommendations on a streaming service—that’s the result of a knowledge graph. 

Of course, the emergence of AI makes the existence of a knowledge graph all the important. The more you’re able to identify, organize, and put your organization’s content into a structured environment, the more easily you’re able to use it in more interesting ways. Help center content is just one example. 

Any organization with structured data has the opportunity to energize the delivery of that content with AI. Now, isn’t that something a content strategist, content designer, or even a UX writer might want to be involved with? We think so. 

We should be clear: building a knowledge graph is no small feat. It’s a cross-departmental journey that requires getting a lot of stakeholders involved. But the sooner content designers and UX writers get involved in these types of projects, the more likely they’ll be seen as leaders in the AI space.

One key step you can take in this area is creating chat functionality to explore any database of structured content. 

The emergence of AI model design

Is AI changing the role of content design and UX writing itself, or just expanding it into new territory?

Is it creating new roles entirely that existing job descriptions can’t describe?

All of the above, really. Although this is a relatively new field and there’s still much to be discovered, some content designers are finding new roles in helping train custom Large Language Models on different forms of content.

Some of these content designers are now giving themselves a new name: AI Model Designers.

This is still an emerging field, and there is much to learn. But there are already some fascinating developments.

Laura Costantino, a Content Designer at Google, recently described how they are working on Large Language Models in their work as a content designer:

“So training data for a Large Language Model, of course, we’re talking about volume of data that is really hard to wrap our heads around, and two techniques—and one in particular—that we’ve been using are fine-tuning and reinforced learning. And there are all sorts of methodologies that are used and most methodologies require to look at content at scale, like ingest.”

“And some of that of course, is the work of a content designer, but I think here it becomes even a little bit more not just the guidelines in terms of style and voice and tone, but also operationally, how do we make sure that creating content at scale can work for the team to a scale that is big enough that it helps training the model?”

Another example is Lisa Jennings Young, the Head of Content Design for Microsoft Teams. She says the team is preparing for a situation where AI could help write between 30% to 80% of a help article. What happens with the rest of that time?

“And because there’s so much more than we could ever possibly do, what we are excited about is it will give us more time to do impactful, deep work, whether it’s more A/B testing, experimentation, more research, journey maps, and those kinds of things.”

These two examples (and there are plenty more) demonstrate that content design is changing drastically as AI takes hold. Content designers and UX writers should start contemplating how AI intersects with their work—because it’s not about “if” the job will change. The question is “when?”

How will content design roles change for AI?

The real answer to this question is “we don’t know.” But one key point we’d consider is that content designers are already starting to do this work.

Chelsea Larsson is the Head of Experience Design at Expedia. She spoke to us last year about how content design is contributing to the company’s AI models:

“You will get much more effective generating responses if you have a big corpus of work of really good content and written in the way that you want written, in the tone that you want, because that will give them those patterns to recognize and then to replicate.”

“For content designers to lead in this space, they really can’t be afraid to get technical. But the technical parts of this are all built on how language is structured. And this is how we already think. So, we don’t have to get our foot in the door, our foot’s already in the door. We just need to walk through it.”

“And that, I hope, is reassuring to people who feel like this is a new technology that they have to learn.”

We can’t predict the future, but it looks as though the relationship between AI and content design is already in a good position.

Some AI content terms strategists should know

Content operations in the age of artificial intelligence (AI) are characterized by the integration of AI technologies into various aspects of the content lifecycle including creation, management, distribution, and analysis.

Here are some examples about content operations in relation to AI and how operations can be supported by AI too:

Automated content creation: AI technologies such as natural language generation (NLG) can automate the process of generating content. This can increase efficiency and reduce the time and resources required for content creation. There’s a significant caveat to using AI for content creation though. That’s to be sure that the content created is accurate, in the required style, compliant where needed and is clear. Care must be taken when writing prompts for AI and a person will have additional context and knowledge about business goals and user needs that AI will not.

Content optimization: AI tools can analyze data to optimize content for specific audiences, platforms, and formats. AI-powered analytics can identify trends and preferences among target audiences, enabling content creators to tailor their content accordingly. As with any data set, contextual analysis is needed to draw meaningful insights from the data available.

Personalization: AI algorithms can personalize content experiences for users based on their preferences, behavior, and demographics. This personalization can improve user engagement and satisfaction by delivering relevant content recommendations and suggestions.

We cover personalization in more detail in our Marketing Writing for UX course.

Content curation: AI-powered content curation tools can identify relevant and high-quality pieces for a particular audience or topic. 

Automated distribution: AI can optimize the distribution of content across various channels, including social media, email, and websites. AI algorithms can determine the best timing, frequency, and channels for distributing content to maximize reach and engagement.

Performance analytics: AI-driven analytics tools can provide insights into content performance, including metrics such as engagement, conversion rates, and ROI (Return on Investment). This data can inform content strategy and optimization efforts.

Natural Language Processing (NLP): NLP technologies enable AI systems to understand and process human language, facilitating tasks such as sentiment analysis, topic modeling, and language translation. These capabilities enhance content operations by extracting meaningful insights from textual content.

AI helps with content operations by automating tasks, personalizing experiences, optimizing performance, and enabling deeper insights into audience preferences and behavior. As the capabilities of AI continue to evolve and emerge, caution should be taken when using AI and understanding where it can enhance and support operations in a meaningful way without losing clarity of content, audience understanding and strategic thinking. 

Where will AI and content design go next?

Clearly, content design and UX writing are changing. It’s too early to say where AI will take both disciplines, but there’s good reason to suggest we’re on a positive path.

Content designers are using AI tools every day. They are feeling positive about the future, and there are content designers working on AI models at a variety of organizations.

The AI age of content design has arrived—and the industry needs to adapt. 

Free UX content resources in your inbox!

Get our weekly Dash newsletter packed with links, regular updates with resources, discounts, and more.