NExT-Chat: An LMM for Chat, Detection and Segmentation

This paper presents NExT-Chat, a large multimodal model for chat, detection, and segmentation.

huggingface.co/papers/2311.04498
NExT-Chat: An LMM for Chat, Detection and Segmentation homepage screenshot

About NExT-Chat: An LMM for Chat, Detection and Segmentation

NExT-Chat is a large multimodal model (LMM) described in an arXiv paper, with a Hugging Face paper page as the source. Its main approach, called pixel2emb, has the model output location embeddings that separate decoders convert into locations such as bounding boxes or masks. The paper says this lets the model handle object location tasks within multimodal conversations, and that it can reuse existing methods from detection and segmentation. The stated tasks are visual grounding, region captioning, and grounded reasoning.

The paper reports that pixel2emb performs better than existing state-of-the-art approaches in both location input and output tasks under fair comparison, in scenarios with limited resources. The page is a research paper listing and does not describe a product, access options, or pricing.

Key features

  • Outputs location embeddings that separate decoders turn into bounding boxes or masks
  • Supports location formats such as bounding boxes and masks within multimodal conversations
  • Handles visual grounding, region captioning, and grounded reasoning tasks
  • Applies existing detection and segmentation practices through embedding-based location modeling
Best for: Researchers working on multimodal models, visual grounding, or region-level image understanding.
Good for
  • Reports better performance than existing state-of-the-art approaches in limited-resource scenarios under fair comparison
  • Allows different location formats, such as bounding boxes and masks, to be used in conversations
  • Lets existing detection and segmentation practices be used for localization tasks
Watch out for
  • The reported performance results are stated for limited-resource scenarios; the page does not give results for other settings
  • The page is a research paper listing and does not describe a product, access method, or usage terms

NExT-Chat: An LMM for Chat, Detection and Segmentation at a glance

Type
Website
Pricing
Not confirmed. Check the site
Platforms
Web
Open source
Not stated
Last checked
Oct 6, 2026 · Online when checked
Listed
Oct 5, 2026

NExT-Chat: An LMM for Chat, Detection and Segmentation alternatives

Tools and sites you could use instead, ranked by how closely they match.

All 24 alternatives

Popular in Models & ML Research

Well-known listings from the same topic.

All 69

NExT-Chat: An LMM for Chat, Detection and Segmentation FAQ

What is the pixel2emb method?
It is a method in which the large multimodal model outputs location embeddings, which are then decoded by different decoders into location formats such as bounding boxes and masks.
Does the page link to code?
The page includes a link to a GitHub repository.
Is something wrong or out of date?

Tell us and an editor will check it.

Own this website?

Claim the listing and suggest updates yourself. An editor checks every claim first.

Claim this listing