> ## Documentation Index
> Fetch the complete documentation index at: https://hub.hcompany.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> H Platform has four products: the Agents API (managed computer-use agents, base URL https://agp.eu.hcompany.ai/api/v2 or https://agp.hcompany.ai/api/v2 for the US), the Models API (OpenAI-compatible Holo vision-language models at https://api.hcompany.ai/v1), HoloDesktop CLI (Holo on the user's own desktop), and HoloTab (a free no-code Chrome extension that runs Holo in the user's browser, with recordable routines and schedules).
> Authenticate with a bearer API key from the HAI_API_KEY environment variable. SDKs: `pip install hai-agents` (Python, `from hai_agents import Client`) and `npm install hai-agents` (TypeScript, `import { HaiAgentsClient } from "hai-agents"`). CLI: `hai`.
> Agents do work in a browser or on a desktop; describe the task as an imperative instruction. To run a task quickly, prefer the pre-built agent `h/web-surfer-flash`. Read results from the session's `latest_answer` after it reaches a terminal status.
> Sessions are the unit of work; wait for a terminal status (completed, failed, timed_out, interrupted) before reading the answer. Use webhooks or the `changes` long-poll endpoint to follow progress.

# Element localization

> GUI grounding with Holo: send a screenshot and an element description, get click coordinates back.

export const LocalizationDemo = () => {
  const SRC = "/images/models-api/screenspot-pro-blender.webp";
  const IMG_W = 2560;
  const IMG_H = 1440;
  const ASPECT = IMG_H / IMG_W;
  const ZOOM = 10;
  const BOX = 64;
  const RED = "#e11d48";
  const targets = [{
    label: "Hide icing",
    element: "Hide the icing layer.",
    x: 984,
    y: 115,
    truth: [2511, 158, 2527, 173]
  }, {
    label: "Donut",
    element: "the donut in the 3D viewport",
    x: 513,
    y: 200
  }, {
    label: "Join Geometry",
    element: "the Join Geometry node header",
    x: 530,
    y: 516
  }, {
    label: "Density Factor",
    element: "the Density Factor field in the Distribute Points on Faces node",
    x: 266,
    y: 800
  }, {
    label: "Play",
    element: "the Play button in the timeline",
    x: 420,
    y: 895
  }, {
    label: "Render menu",
    element: "the Render menu in the top menu bar",
    x: 46,
    y: 24
  }];
  const [i, setI] = useState(0);
  const t = targets[i];
  const absX = Math.floor(t.x / 1000 * IMG_W);
  const absY = Math.floor(t.y / 1000 * IMG_H);
  const truth = t.truth && ({
    x: t.truth[0] / IMG_W * 1000,
    y: t.truth[1] / IMG_H * 1000,
    w: (t.truth[2] - t.truth[0]) / IMG_W * 1000,
    h: (t.truth[3] - t.truth[1]) / IMG_H * 1000
  });
  const center = {
    transform: "translate(-50%, -50%)"
  };
  const right = t.x < 500;
  const below = t.y < 620;
  const pop = {
    left: `${t.x / 10}%`,
    top: `${t.y / 10}%`,
    transform: `translate(${right ? "18px" : "calc(-100% - 18px)"}, ${below ? "18px" : "calc(-100% - 18px)"})`
  };
  const zoomStyle = {
    width: BOX,
    height: BOX,
    backgroundImage: `url(${SRC})`,
    backgroundSize: `${ZOOM * BOX}px auto`,
    backgroundPosition: `${BOX / 2 - t.x / 1000 * ZOOM * BOX}px ${BOX / 2 - t.y / 1000 * ZOOM * BOX * ASPECT}px`,
    backgroundRepeat: "no-repeat"
  };
  return <div className="not-prose my-6 overflow-hidden rounded-xl border border-zinc-200 bg-white dark:border-zinc-800 dark:bg-zinc-950">
      <div className="grid gap-1 border-b border-zinc-200 px-3 py-2 dark:border-zinc-800" style={{
    gridTemplateColumns: `repeat(${targets.length}, minmax(0, 1fr))`
  }}>
        {targets.map((tg, k) => <button key={tg.label} type="button" onClick={() => setI(k)} className={`${k === i ? "rounded-lg px-2 py-2 text-center text-sm font-medium leading-tight transition-colors bg-zinc-100 text-zinc-900 dark:bg-zinc-800 dark:text-zinc-50" : "rounded-lg px-2 py-2 text-center text-sm font-medium leading-tight transition-colors text-zinc-500 hover:bg-zinc-50 hover:text-zinc-800 dark:text-zinc-400 dark:hover:bg-zinc-900 dark:hover:text-zinc-200"}`}>
            {tg.label}
          </button>)}
      </div>

      <div className="relative">
        <img src={SRC} alt="Blender with a donut scene, geometry nodes editor, and outliner" className="block w-full" />
        <svg className="pointer-events-none absolute inset-0 h-full w-full" viewBox="0 0 1000 1000" preserveAspectRatio="none">
          <line x1={t.x} y1="0" x2={t.x} y2="1000" stroke={RED} strokeWidth="1" strokeDasharray="3 5" vectorEffect="non-scaling-stroke" opacity="0.7" />
          <line x1="0" y1={t.y} x2="1000" y2={t.y} stroke={RED} strokeWidth="1" strokeDasharray="3 5" vectorEffect="non-scaling-stroke" opacity="0.7" />
          {truth && <rect x={truth.x} y={truth.y} width={truth.w} height={truth.h} fill="none" stroke="#fff" strokeWidth="1.5" vectorEffect="non-scaling-stroke" />}
        </svg>
        <span className="pointer-events-none absolute rounded-full" style={{
    left: `${t.x / 10}%`,
    top: `${t.y / 10}%`,
    width: 22,
    height: 22,
    border: `1.5px solid ${RED}`,
    ...center
  }} />
        <span className="pointer-events-none absolute rounded-full" style={{
    left: `${t.x / 10}%`,
    top: `${t.y / 10}%`,
    width: 8,
    height: 8,
    background: RED,
    ...center
  }} />

        <div className="pointer-events-none absolute flex w-max max-w-[340px] gap-3 rounded-lg border border-zinc-200 bg-white/95 p-3 shadow-lg backdrop-blur dark:border-zinc-700 dark:bg-zinc-950/95" style={pop}>
          <div className="relative shrink-0 overflow-hidden rounded-md border border-zinc-200 dark:border-zinc-700" style={zoomStyle}>
            <span className="absolute left-1/2 top-1/2 rounded-full" style={{
    width: 12,
    height: 12,
    border: `1.5px solid ${RED}`,
    ...center
  }} />
            {truth && <span className="absolute" style={{
    border: "1px solid #fff",
    left: BOX / 2 + (truth.x - t.x) / 1000 * ZOOM * BOX,
    top: BOX / 2 + (truth.y - t.y) / 1000 * ZOOM * BOX * ASPECT,
    width: truth.w / 1000 * ZOOM * BOX,
    height: truth.h / 1000 * ZOOM * BOX * ASPECT
  }} />}
          </div>
          <div className="flex min-w-0 flex-col gap-1.5">
            <div className="text-sm leading-5 text-zinc-800 dark:text-zinc-200">{t.element}</div>
            <span className="w-fit rounded-md bg-zinc-100 px-2 py-0.5 font-mono text-xs text-zinc-800 dark:bg-zinc-800 dark:text-zinc-100">{`{"x": ${t.x}, "y": ${t.y}}`}</span>
            <div className="text-xs text-zinc-500 dark:text-zinc-400">
              {absX}, {absY} px on {IMG_W} × {IMG_H}
              {truth && <span> · inside ground truth</span>}
            </div>
          </div>
        </div>
      </div>

      <div className="flex flex-wrap justify-between gap-x-6 gap-y-1 border-t border-zinc-200 px-5 py-3 text-sm text-zinc-500 dark:border-zinc-800 dark:text-zinc-400">
        <span>
          <a href="https://huggingface.co/datasets/likaixin/ScreenSpot-Pro" target="_blank" rel="noreferrer" className="underline decoration-zinc-300 underline-offset-2 hover:text-zinc-800 dark:decoration-zinc-600 dark:hover:text-zinc-200">ScreenSpot-Pro</a> · Blender · {IMG_W} × {IMG_H}
        </span>
        <span>Holo3.1 35B-A3B</span>
      </div>
    </div>;
};

<LocalizationDemo />

## Localize an element

Single turn, no history, no thinking: pass the screenshot (URL or base64 data URI) and a description of the element, and ask for JSON. Holo3 and Holo3.1 both support it. Client setup is in the [Quickstart](/models-api/quickstart).

<CodeGroup>
  ```python Python theme={"system"}
  from pydantic import BaseModel, Field

  MODEL_NAME = "holo3-1-35b-a3b"
  SCREENSHOT_URL = "https://your-host/screenshot.png"  # or "data:image/png;base64,..."
  SCREENSHOT_WIDTH, SCREENSHOT_HEIGHT = 1280, 720
  ELEMENT = "the 'Sign in' button in the top-right corner"

  class VisualLocalizerOutput(BaseModel):
      x: int = Field(ge=0, le=1000, description="X coordinate as integer in [0, 1000]")
      y: int = Field(ge=0, le=1000, description="Y coordinate as integer in [0, 1000]")

  schema = VisualLocalizerOutput.model_json_schema()

  prompt = (
      "Localize an element on the GUI image according to the provided target "
      "and output a click position.\n"
      f" * You must output a valid JSON following the format: {schema}\n"
      f" Your target is:\n{ELEMENT}"
  )

  response = client.chat.completions.create(
      model=MODEL_NAME,
      messages=[{
          "role": "user",
          "content": [
              {"type": "image_url", "image_url": {"url": SCREENSHOT_URL}},
              {"type": "text", "text": prompt},
          ],
      }],
      extra_body={
          "structured_outputs": {"json": schema},
          "chat_template_kwargs": {"enable_thinking": False},
      },
      temperature=0.0,
  )

  point = VisualLocalizerOutput.model_validate_json(response.choices[0].message.content)
  abs_x = int(point.x / 1000 * SCREENSHOT_WIDTH)
  abs_y = int(point.y / 1000 * SCREENSHOT_HEIGHT)
  print(f"Click at ({abs_x}, {abs_y})")
  ```

  ```typescript TypeScript theme={"system"}
  const MODEL_NAME = "holo3-1-35b-a3b";
  const SCREENSHOT_URL = "https://your-host/screenshot.png"; // or "data:image/png;base64,..."
  const SCREENSHOT_WIDTH = 1280;
  const SCREENSHOT_HEIGHT = 720;
  const ELEMENT = "the 'Sign in' button in the top-right corner";

  const schema = {
    type: "object",
    properties: {
      x: { type: "integer", minimum: 0, maximum: 1000, description: "X coordinate as integer in [0, 1000]" },
      y: { type: "integer", minimum: 0, maximum: 1000, description: "Y coordinate as integer in [0, 1000]" },
    },
    required: ["x", "y"],
  };

  const prompt =
    "Localize an element on the GUI image according to the provided target " +
    "and output a click position.\n" +
    ` * You must output a valid JSON following the format: ${JSON.stringify(schema)}\n` +
    ` Your target is:\n${ELEMENT}`;

  const response = await client.chat.completions.create({
    model: MODEL_NAME,
    messages: [
      {
        role: "user",
        content: [
          { type: "image_url", image_url: { url: SCREENSHOT_URL } },
          { type: "text", text: prompt },
        ],
      },
    ],
    temperature: 0.0,
    // structured_outputs and chat_template_kwargs are H-specific, passed through in the request body
    ...({
      structured_outputs: { json: schema },
      chat_template_kwargs: { enable_thinking: false },
    } as any),
  });

  const point = JSON.parse(response.choices[0].message.content!) as { x: number; y: number };
  const absX = Math.round((point.x / 1000) * SCREENSHOT_WIDTH);
  const absY = Math.round((point.y / 1000) * SCREENSHOT_HEIGHT);
  console.log(`Click at (${absX}, ${absY})`);
  ```
</CodeGroup>

Coordinates come back as integers in `[0, 1000]`, normalized to the image you sent. Scale against the same image bytes you sent. See [Coordinates](/models-api/build-an-agent/core-concepts#coordinates-in-0-1000).

## Common pitfalls

| Symptom                                 | Cause                                               | Fix                                                                             |
| --------------------------------------- | --------------------------------------------------- | ------------------------------------------------------------------------------- |
| Click lands slightly off                | Screenshot was resized or cropped after the request | Scale with the dimensions of the exact bytes you sent                           |
| Click lands far off on a Retina display | Mixing CSS pixels and device pixels                 | Pick one unit and stay in it end to end                                         |
| Wrong element picked                    | Description is ambiguous ("the button")             | Name the label, position, and neighbors ("the blue Save button below the form") |
| Slow or verbose response                | Thinking left on                                    | Set `enable_thinking=False` and `temperature=0.0`                               |

## Next steps

<CardGroup cols={3}>
  <Card title="Build an agent" icon="arrows-rotate" href="/models-api/build-an-agent/core-concepts">
    How to use Holo in your computer-use harness.
  </Card>

  <Card title="API reference" icon="code" href="/models-api/api-reference">
    Endpoint, models, parameters, and limits.
  </Card>

  <Card title="Quickstart" icon="rocket" href="/models-api/quickstart">
    Back to setup and your first call.
  </Card>
</CardGroup>