The integration between large language models(LLMs) and websites is one of the areas that can unlock a new wave of LLM-powered applications. LLMs have demonstrated remarkable proficiency in a wide array of natural language tasks, ranging from basic arithmetic and logical reasoning to more complex challenges such as commonsense understanding, question answering, and even interactive decision-making. Augmenting these capabilities with web navigation results in a very powerful combination. Recently, Google DeepMind unveiled Web Agent, an LLM-driven autonomous agent capable of navigating real websites based on user instructions.

The real-world implementation of web navigation has posed unique challenges, including:...