darknet_05_process_videos_multithreaded.cpp File Reference

This application will process one or more videos as fast as possible using multiple threads and save a new output video to disk. More...

#include "darknet.hpp"
#include "darknet_image.hpp"
#include <set>
#include <thread>

Include dependency graph for darknet_05_process_videos_multithreaded.cpp:

Classes
struct	Frame
	Everything we know about a specific video frame is stored in one of these objects. More...

Functions
void	detection_thread (size_t &total_objects_found)
int	main (int argc, char *argv[])
void	output_thread (cv::VideoWriter &out)
void	resize_thread ()

Variables
bool	all_threads_must_exit = false
	if something goes wrong, this flag gets set to true
size_t	expected_next_index = 0
	keep track of which frame has been written to disk
std::set< Frame >	frames_waiting_for_output
	once a frame has been predicted, it is stored here
std::set< Frame >	frames_waiting_for_prediction
	once a frame has been resized, it is stored here
std::set< Frame >	frames_waiting_for_resize
	once a frame is read from the video, it is stored here
Darknet::NetworkPtr	net = nullptr
	Darknet/YOLO neural network pointer.
cv::Size	network_dimensions
	dimensions of the neural network that was loaded
size_t	output_thread_starved = 0
std::chrono::high_resolution_clock::duration	output_work_duration
	amount of time spent on the output video
size_t	predict_thread_starved = 0
std::chrono::high_resolution_clock::duration	predict_work_duration
	amount of time spent predicting frames
size_t	reader_must_pause = 0
std::chrono::high_resolution_clock::duration	reader_work_duration
	amount of time spent reading frames
size_t	resize_thread_starved = 0
std::chrono::high_resolution_clock::duration	resize_work_duration
	amount of time spent resizing frames
std::chrono::high_resolution_clock::duration	wait_threads_duration
	amount of time spent waiting for other threads to finish running
std::mutex	waiting_for_output
	mutex to protect access to frames_waiting_for_output
std::mutex	waiting_for_prediction
	mutex to protect access to frames_waiting_for_prediction
std::mutex	waiting_for_resize
	mutex to protect access to frames_waiting_for_resize
size_t	waiting_for_threads = 0

Detailed Description

This application will process one or more videos as fast as possible using multiple threads and save a new output video to disk.

The results are not shown to the user. Call it like this:

darknet_05_process_videos_multithreaded LegoGears DSCN1582A.MOV

The output should be similar to this:

processing DSCN1582A.MOV:
-> total number of CPUs ..... 16
-> threads for this video ... 4
-> neural network size ...... 224 x 160 x 3
-> input video dimensions ... 640 x 480
-> input video frame count .. 1230
-> input video frame rate ... 29.970030 FPS
-> input video length ....... 41041 milliseconds
-> output filename .......... DSCN1582A_output.m4v
-> total frames processed ... 1230
-> time to process video .... 1719 milliseconds
-> processed frame rate ..... 715.532286 FPS
-> total objects found ...... 6189
-> average objects/frame .... 5.031707

Function Documentation

detection_thread()

void detection_thread

(

size_t &

total_objects_found

)

Here is the call graph for this function:

Here is the caller graph for this function:

main()

int main	(	int	argc,
		char *	argv[]
	)

Note

This gets complicated. There are many fourcc values (codecs) and many container types (file extensions). No single value will work for absolutely everyone. Some of the common ones are:

• MJPG .avi -> simple, widely supported, creates large files
• XVID .avi -> generic, widely used, creates large files
• DIVX .avi -> similar to xvid
• H264 .mp4 -> high efficiency, small files, requires ffmpeg?
• X264 .mp4 -> high efficiency, small files, requires ffmpeg?
• avc1 .mp4 -> high efficiency, small files, requires ffmpeg?
• MP4V .mp4 -> common, large files, what Darknet/YOLO used up until v5.x
• PIM1 .avi -> older, universal

Darknet/YOLO used to create .m4v files with the fourcc value "mp4v". Turns out that .m4v is an Apple-specific extension. In V6+, the file extension was changed to .mp4, and the fourcc was changed to "avc1". This creates much smaller video files than the previous setting.

Also note that the higher the compression, the longer it takes to encode the video. Remember to modify the output filename above if you are trying different fourcc values.

Here is the call graph for this function:

output_thread()

void output_thread

(

cv::VideoWriter &

out

)

Here is the caller graph for this function:

resize_thread()

void resize_thread

(

)

Here is the call graph for this function:

Here is the caller graph for this function:

Variable Documentation

all_threads_must_exit

bool all_threads_must_exit = false

if something goes wrong, this flag gets set to true

expected_next_index

size_t expected_next_index = 0

keep track of which frame has been written to disk

frames_waiting_for_output

std::set<Frame> frames_waiting_for_output

once a frame has been predicted, it is stored here

frames_waiting_for_prediction

std::set<Frame> frames_waiting_for_prediction

once a frame has been resized, it is stored here

frames_waiting_for_resize

std::set<Frame> frames_waiting_for_resize

once a frame is read from the video, it is stored here

net

Darknet::NetworkPtr net = nullptr

Darknet/YOLO neural network pointer.

network_dimensions

cv::Size network_dimensions

dimensions of the neural network that was loaded

output_thread_starved

size_t output_thread_starved = 0

output_work_duration

std::chrono::high_resolution_clock::duration output_work_duration

amount of time spent on the output video

predict_thread_starved

size_t predict_thread_starved = 0

predict_work_duration

std::chrono::high_resolution_clock::duration predict_work_duration

amount of time spent predicting frames

reader_must_pause

size_t reader_must_pause = 0

reader_work_duration

std::chrono::high_resolution_clock::duration reader_work_duration

amount of time spent reading frames

resize_thread_starved

size_t resize_thread_starved = 0

resize_work_duration

std::chrono::high_resolution_clock::duration resize_work_duration

amount of time spent resizing frames

wait_threads_duration

std::chrono::high_resolution_clock::duration wait_threads_duration

amount of time spent waiting for other threads to finish running

waiting_for_output

std::mutex waiting_for_output

mutex to protect access to frames_waiting_for_output

waiting_for_prediction

std::mutex waiting_for_prediction

mutex to protect access to frames_waiting_for_prediction

waiting_for_resize

std::mutex waiting_for_resize

mutex to protect access to frames_waiting_for_resize

waiting_for_threads

size_t waiting_for_threads = 0