Video segmentation with Python using SAM2! - 파이썬 SAM2 실습 : 비디오에서 누끼따기!
🦖(ENGLISH) Python SAM2 Practice: Background Removal from Video!
In this post, we’ll walk through a practical use case of SAM2, the successor to Meta’s Segment Anything Model (SAM).
SAM2 is already supported by the popular ultralytics library, making it incredibly simple to use!
That means — anyone can try it, easily and quickly! 🚀
🧱 1. Installing ultralytics
- Already installed? Then skip this step!
1
pip install ultralytics
📦 2. Download & Load the Model
1
2
3
4
5
6
7
from ultralytics import SAM
import cv2
import matplotlib.pyplot as plt
import numpy as np
# Load the SAM2 model
model = SAM("sam2.1_b.pt")
We also loaded a few additional packages for visualization.
🧊 3. Run Image Segmentation!
Let’s segment a dog image using a bounding box prompt:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
img_name = "dog.jpg"
my_bboxes = [1430.2, 828, 4471.9, 3836.4]
# Run inference with a box prompt [x_min, y_min, x_max, y_max]
results = model(img_name, bboxes=my_bboxes)
# Load the original image (for visualization)
image = cv2.imread(img_name)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# Show results
plt.figure(figsize=(10, 10))
plt.imshow(image_rgb)
# Overlay mask
for result in results:
if result.masks is not None:
mask = result.masks.data[0].cpu().numpy()
plt.imshow(mask, alpha=0.5, cmap='jet')
# Draw bounding box
rect = plt.Rectangle((my_bboxes[0], my_bboxes[1]), my_bboxes[2] - my_bboxes[0], my_bboxes[3] - my_bboxes[1],
linewidth=2, edgecolor='red', facecolor='none', label=f'my_bboxes {my_bboxes}')
plt.gca().add_patch(rect)
# Final touches
plt.title(f"SAM2 Segmentation with Box Prompt on {img_name}")
plt.legend()
plt.axis('off')
plt.show()
# Optional: Print extra info
print("Segmentation Result:")
print(f"Number of masks: {len(results[0].masks.data)}")
print(f"Mask shape: {results[0].masks.data[0].shape}")
Pretty impressive, right?
But to be fair — this is something even SAM1 handled quite well 😎
🚀 4. Run Video Segmentation!
Now let’s try something SAM1 couldn’t do: video segmentation — a core strength of SAM2!
I used a highway CCTV video and provided a single point prompt at (405, 205) in the first frame where a car is visible:
1
2
3
4
5
6
7
8
from ultralytics.models.sam import SAM2VideoPredictor
# Create SAM2VideoPredictor
overrides = dict(conf=0.25, task="segment", mode="predict", imgsz=1024, model="sam2_b.pt")
predictor = SAM2VideoPredictor(overrides=overrides)
# Run inference with a single point
results = predictor(source="street.mp4", points=[405, 205], labels=[1])
I can’t upload videos here, but as shown in the screenshots below,
SAM2 was able to segment the car only while it was visible — and stopped at the right moment. Very smart!
🎉 Final Thoughts
I was genuinely impressed not only by SAM2’s ability to segment in videos,
but also by how well it tracked objects over time with minimal input.
Highly recommend trying this out — especially for any video segmentation tasks!
🦖(한국어) 파이썬 SAM2 실습 : 비디오에서 누끼따기!
이번 포스팅은 SAM의 후속 모델인 SAM2 실습입니다!
SAM2는 유명한 모델이어서 그런지, ultralytics 의 패키지로 쉽게 사용할수 있었습니다!! 그래서 모두 간단히, 쉽게 사용가능합니다!!!
🧱 1. ultralytics 설치
- 이미 설치된 분이라면 스킵쓰!
1
pip install ultralytics
📦 2. 모델 다운 및 로드!
1
2
3
4
5
6
7
8
from ultralytics import SAM
import cv2
import matplotlib.pyplot as plt
import numpy as np
# 모델 로드
model = SAM("sam2.1_b.pt")
시각화를 위해 그외 패키지도 로드해주었습니다!
🧊 3. 이미지 Segment 실행!!
기존 사용했던 강아지 이미지를, bbox와 함께 segment해보았습니다!!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
img_name = "dog.jpg"
my_bboxes=[1430.2, 828, 4471.9, 3836.4]
# 박스 프롬프트로 추론 ([x_min, y_min, x_max, y_max])
results = model(img_name, bboxes=my_bboxes)
# 원본 이미지 로드 (시각화를 위해)
image = cv2.imread(img_name)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB) # BGR -> RGB 변환
# 결과 시각화
plt.figure(figsize=(10, 10))
plt.imshow(image_rgb)
# 마스크 오버레이
for result in results:
if result.masks is not None:
mask = result.masks.data[0].cpu().numpy() # 첫 번째 마스크 추출
plt.imshow(mask, alpha=0.5, cmap='jet') # 마스크를 반투명하게 표시
# 박스 프롬프트 표시
rect = plt.Rectangle((my_bboxes[0], my_bboxes[1]), my_bboxes[2] - my_bboxes[0], my_bboxes[3] - my_bboxes[1],
linewidth=2, edgecolor='red', facecolor='none', label=f'my_bboxes {my_bboxes}')
plt.gca().add_patch(rect)
# 제목 및 설정
plt.title(f"SAM2 Segmentation with Box Prompt on {img_name}")
plt.legend()
plt.axis('off')
plt.show()
# 추가 정보 출력 (선택 사항)
print("Segmentation Result:")
print(f"Number of masks: {len(results[0].masks.data)}")
print(f"Mask shape: {results[0].masks.data[0].shape}")
참 잘되죠~ 그런데 이건 SAM도 잘하건건데!?
🚀 4. 비디오 Segment 실행!!
그래서, 이번엔 SAM2의 특징인!
비디오의 segment도 진행해보았습니다!
저는 고속도로의 CCTV영상을 바탕으로 진행했구요! 첫 프래임에서 차가 있는 곳의 위치(405,205)를 프롬포트로 제공했습니다!
1
2
3
4
5
6
7
8
from ultralytics.models.sam import SAM2VideoPredictor
# Create SAM2VideoPredictor
overrides = dict(conf=0.25, task="segment", mode="predict", imgsz=1024, model="sam2_b.pt")
predictor = SAM2VideoPredictor(overrides=overrides)
# Run inference with single point
results = predictor(source="street.mp4", points=[405, 205], labels=[1])
동영상을 올릴순 없지만!!
아래 스크린샷같이 차가 사라지는 시점까지만 딱!!segment를 정말 잘하더라구요!!
🎉 마무리
동영상의 segmentation에 더하여, 저는 Tracking이 이렇게 잘된다는것이 너무 인상적이었습니다!