Segment anything model (SAM)
Segment anything model (SAM) allows you to automatically create labels around distinct features in all supported file formats. Read more about auto segmentation advances in our explainer blog post here.
Video Tutorial - SAM
The steps below show and describe how to use Encord's new automated segmentation feature.
SAM is only available for ontologies that have the 'Polygon', or 'Bounding box' annotation types.
1. Start labeling using SAM
Click the 
icon within the Polygon, or Bounding box class when viewing a task in the label editor to make the SAM pop-up appear.
The SAM pop-up shows instructions on how to use the tool, as well as updates while the tool is running.
Use the Shift + A keyboard shortcut to toggle SAM mode.
Use the slider in the pop-up to prioritize either the speed, or quality of your SAM labels. Moving the slider towards the 'Faster' end allows users working on high-resolution images or frames to increase the pace at which labels are generated, by downsampling the image before it is sent to the server.
| Slider position | Quality |
|---|---|
| Faster | 20% |
| Default | 60% |
| Higher quality | 100% |
SAM mode will remain active when switching to a different annotation class, and will only be deactivated by clicking the icon again.
2. Using SAM to segment a frame or image
Click the area on the image or frame you would like to segment - the pop-up will indicate that auto-segmentation is running, as shown below.
You can also click and drag your cursor to select the part of an image you would like to segment.
3. Including / Excluding areas from selection
Once the prediction has run, a part of the image or frame will be highlighted in blue.
- You can add to the selected area by left clicking the part of the image you'd like to add to the label. This is shown in the video below.
- You can exclude parts of the selected area by right-clicking the part you'd like to exclude from the label.
- If you've made a mistake and want to re-start, simply click Reset on the SAM pop-up to start again.
4. Confirming the label
Once the correct section is highlighted, click Apply Label on the SAM pop-up, or press Enter on your keyboard to confirm the selection. The result will be a label outlining the area that was highlighted - the label shape will depend on whether the selection was made using a bounding box, or a polygon.
Repeat the above process for all the labels in the frame or image.
SAM works in conjunction with the 'Permanent drawing' setting which can increase the speed at which SAM labels are created in video frames.