Summary of Malleable software in the age of LLMs

  • geoffreylitt.com
  • Article
  • Summarized Content

    Introduction: ChatGPT's Impact on Software Development

    The article discusses the potential impact of large language models (LLMs) like ChatGPT on the creation and distribution of software, particularly in empowering end-user programming.

    • OpenAI's release of GPT-4 and ChatGPT plugins has sparked discussions about how LLMs will affect software development.
    • While LLMs will undoubtedly make professional developers more productive, the author believes they could also enable end-user programming, allowing regular computer users to develop small software tools from scratch or modify existing software.
    • This could lead to significant changes in when, by whom, and for what purpose software is created.

    ChatGPT's Coding Capabilities

    The article explores the current coding capabilities of ChatGPT and GPT-4, highlighting both impressive examples and limitations.

    • GPT-4 has shown impressive coding abilities, such as generating a 3D video game from a zero-shot prompt.
    • However, it still struggles with relatively simple algorithm problems and can get confused when developing certain interactions.
    • The author suggests that iteration and clarification are natural parts of the process with LLMs, and their coding abilities are rapidly improving with each generation.

    Chat vs. Direct Manipulation User Interfaces

    The article discusses the limitations of chat interfaces compared to direct manipulation user interfaces (UIs) for certain tasks.

    • Chat interfaces can be slow, imprecise, and require conscious awareness of the thought process, unlike direct manipulation UIs that allow for subconscious flow states and fine control.
    • The author suggests that while ChatGPT is more capable than assistants like Siri, many tasks are better served by graphical user interfaces (GUIs) than chat.
    • Hybrid interaction models, where LLMs help users construct and modify UIs, are proposed as a potential solution.

    Consultants vs. Apps: LLMs' Role in Software Interaction

    The article compares the flexibility of human consultants with the limitations of traditional software applications and explores how LLMs could bridge the gap.

    • Human consultants offer flexibility but are expensive and time-consuming.
    • Traditional apps are faster and cheaper but lack flexibility for tasks outside their scope.
    • LLMs could replace human consultants, offering flexibility at a lower cost, while also enabling users to update and customize applications on-the-fly with their help.

    Computational Media and End-User Programming with ChatGPT

    The article proposes the idea of open-ended computational media, directly learnable and moldable by users, with ChatGPT acting as a collaborator.

    • Spreadsheets are cited as an example of a general computational medium that allows users to flexibly express many tasks.
    • The author suggests that LLMs could play the role of "local developers," helping users create and modify tools within the computational medium while also teaching them to become more self-sufficient over time.
    • This approach combines the benefits of direct manipulation UIs with the flexibility of LLM-assisted tool editing and knowledge transfer.

    On-the-Fly UI Generation with ChatGPT

    The article explores the potential of on-the-fly UI generation with the help of ChatGPT, showcasing examples and discussing challenges.

    • Demos from various sources illustrate the generation of interactive table views, maps, video editing UIs, and forms for server requests using ChatGPT.
    • The author highlights the need for an iterative process between the user and the LLM to create meaningful UIs.
    • Challenges include handling the iterative process smoothly and enabling users to inspect and understand the generated UI components.

    Extensible Software and the Future

    The article concludes by setting the stage for the next discussion on the architectural foundations required to make GUI applications extensible and composable by people using LLMs.

    • The author plans to explore how to enable users to pull apart and recombine UI elements within applications using ChatGPT.
    • This could potentially lead to a future where software is more malleable and customizable by end-users, with the help of LLMs.
    • The article acknowledges the speculative nature of these ideas and the uncertainty surrounding the pace of change, but aims to inspire thinking about how AI could empower end-users in new ways.

    Discover content by category

    Ask anything...

    Sign Up Free to ask questions about anything you want to learn.