Originally posted: 2023-10-19. View source code for this page here.
This post contains some fairly disorganised thoughts and questions on the short term (maybe 0 - 3 years)1 effects of LLMs on knowledge workers.
Many of these effects are already happening, but probably in the early stages of the S curve.
Key to most of these observations is the tension between useful applications of LLMs, and their ability to churn out vacuous content:
Less motivated staff will be able to easily produce superficially impressive work devoid of genuinely useful content. Think: fancy consultancy-style Powerpoints.
The attention of high performing staff will potentially be diverted because they will find it harder to distinguish between genuinely insightful and high effort content, and superficial AI-generated content.
Question: Will the offensive or defensive capabilities of LLMs dominate here? Will most content (especially from colleagues who are less well known, or with weaker reputations) will be passed through an LLM summariser/de-jargoner before being read?
It could go in the other direction: LLMs may become so useful at summarisation and understanding the preferences of their users that much low value corporate information is neutralised.
Question: How will workers signal that content is their original ideas, rather than AI-generated?2
There will be a rebalancing of the most valuable 'core' skills of knowledge workers:
Precise use of language will increase in importance. Most obviously, the ability precisely and creatively to instruct an LLM.
Substance will prevail over style since knowledge workers will use LLMs defensively to filter out jargon and other low-information-value content, in a similar way to how ad-blockers make the modern internet tolerable.
In-person contact will be the best signal of a person's original thinking, so will become even more important to developing a reputation
Assembling the relevant information to a decision will become relatively more important, and the ability to summarise it less so, since managers will often 'chat' with the information rather than want to read a hand-crafted report.
Pulling out a few key insights that managers should want to know will still be important since LLMs will not have the full context for the decision.
Tasks involving lots of 'boilerplate' (whether coding or correspondence) will be largely automated, meaning fewer people can get more done.
Information will become even easier to retrieve, so memory will be less important than the ability to join up ideas
LLMs make mistakes, so critical thinking and the ability to challenge information and find corroboration will still be important.
These are not dramatic changes, since most of these skills are already important to highly effective knowledge workers. But they will further leverage productivity differentials.
Question: To what extent will LLMs disrupt the ability to be successful based on excessive use of jargon/buzzwords and dressing up ideas, as opposed to communicating in plain English and letting the clarity of thinking and quality of ideas do the work
LLMs are already highly effective when used to filter out clickbait from the internet - surely the same will be possible in the workplace.
The effort required by a candidate to create to write a superficially impressive job application, highly tailored to the job description and company will be very low.
As a result, the strength of the signal in job applications will be significantly reduced making sifting much harder. More will have to rest on the interview, which are pretty noisy signals.
Question: Will a fundamental change in recruitment process be necessary? Perhaps a greater emphasis on probationary periods?
Reputation may become even more important.
With a highly effective personal tutor for everyone, motivation and enthusiasm to learn will become even more important. The skill of learning new things quickly will become even more important.
Classroom-based training for most generic skills will be relatively less effective, except for where there's an ulterior purpose (e.g. networking).
Superficial knowledge of programming languages will decline in value substantially, but deep knowledge of languages and the principles of programming will still be just as important.
Junior staff will still be useful, but tenacity (as a skill/behaviour) will increase in importance (the ability to solve the problem the LLM couldn't, which by definition will be the tricky ones.)
Productivity will dramatically increase amongst the most motivated.
The design of software will change. Developers will target building software that is described precisely enough that most people can use it via an LLM (i.e. the LLM can write the code by holding the library in context and thereby understanding how it should be used.)
Complex refactors that change the structure of big codebases will still need to be 'led' by highly experienced humans for a long time - probably at least 3-5 years.
This post was not written by an LLM! However, I did use ChatGPT advanced data analysis to pull the following stats:
I've also used Claude (for its longer context) and Bing chat a bit, and prior to that the earliest reference I can find to using LLMs was Eleuther AI's 6b model in June 2021.
Here is a timeline.
From this experience, my overall sense is that LLMs represent a truly transformational technological breakthrough, similar in scale to the internet, in that they will completely transform the way we interact with information. For example, my use of Google has dramatically diminished since I've started using LLMs.
I'm assuming in the 3 year time horizon, there will be significant increase in the context size for LLMs ↩
It's more subtle than 'an AI was not used in the production of the work', since the AI may genuinely enhance the content e.g. by proofreading and making suggestions. But since AI rarely generates unique insights, it's less good for generating ideas and strategies. ↩