# OpenCV 5 > OpenCV 5.x documentation and tutorials from the opencv/opencv 5.x branch (doc/). ## [OpenCV 5](https://docharvest.github.io/docs/opencv5/) Contents opencv5 OpenCV 5 OpenCV 5 OpenCV 5.x documentation and tutorials from the opencv/opencv 5.x branch (doc/). OpenCV 5.x documentation and tutorials from the opencv/opencv 5.x branch (doc/). - [Faqfaq](/docs/opencv5/faq/) - [Opencv Logoopencv-logo](/docs/opencv5/opencv-logo/) - [Js Tutorialsjs\_tutorials/js\_tutorials](/docs/opencv5/js_tutorials/js_tutorials/) - [Py Tutorialspy\_tutorials/py\_tutorials](/docs/opencv5/py_tutorials/py_tutorials/) - [Tutorialstutorials/tutorials](/docs/opencv5/tutorials/tutorials/) - [Js Table Of Contents Corejs\_tutorials/js\_core/js\_table\_of\_contents\_core](/docs/opencv5/js_tutorials/js_core/js_table_of_contents_core/) - [Js Table Of Contents Dnnjs\_tutorials/js\_dnn/js\_table\_of\_contents\_dnn](/docs/opencv5/js_tutorials/js_dnn/js_table_of_contents_dnn/) - [Js Table Of Contents Guijs\_tutorials/js\_gui/js\_table\_of\_contents\_gui](/docs/opencv5/js_tutorials/js_gui/js_table_of_contents_gui/) - [Js Table Of Contents Imgprocjs\_tutorials/js\_imgproc/js\_table\_of\_contents\_imgproc](/docs/opencv5/js_tutorials/js_imgproc/js_table_of_contents_imgproc/) - [Js Table Of Contents Setupjs\_tutorials/js\_setup/js\_table\_of\_contents\_setup](/docs/opencv5/js_tutorials/js_setup/js_table_of_contents_setup/) - [Js Table Of Contents Videojs\_tutorials/js\_video/js\_table\_of\_contents\_video](/docs/opencv5/js_tutorials/js_video/js_table_of_contents_video/) - [Py Table Of Contents Bindingspy\_tutorials/py\_bindings/py\_table\_of\_contents\_bindings](/docs/opencv5/py_tutorials/py_bindings/py_table_of_contents_bindings/) - [Py Table Of Contents Calib3dpy\_tutorials/py\_calib3d/py\_table\_of\_contents\_calib3d](/docs/opencv5/py_tutorials/py_calib3d/py_table_of_contents_calib3d/) - [Py Table Of Contents Corepy\_tutorials/py\_core/py\_table\_of\_contents\_core](/docs/opencv5/py_tutorials/py_core/py_table_of_contents_core/) - [Py Table Of Contents Featurespy\_tutorials/py\_features/py\_table\_of\_contents\_features](/docs/opencv5/py_tutorials/py_features/py_table_of_contents_features/) - [Py Table Of Contents Guipy\_tutorials/py\_gui/py\_table\_of\_contents\_gui](/docs/opencv5/py_tutorials/py_gui/py_table_of_contents_gui/) - [Py Table Of Contents Imgprocpy\_tutorials/py\_imgproc/py\_table\_of\_contents\_imgproc](/docs/opencv5/py_tutorials/py_imgproc/py_table_of_contents_imgproc/) - [Py Table Of Contents Mlpy\_tutorials/py\_ml/py\_table\_of\_contents\_ml](/docs/opencv5/py_tutorials/py_ml/py_table_of_contents_ml/) - [Py Table Of Contents Objdetectpy\_tutorials/py\_objdetect/py\_table\_of\_contents\_objdetect](/docs/opencv5/py_tutorials/py_objdetect/py_table_of_contents_objdetect/) - [Py Table Of Contents Photopy\_tutorials/py\_photo/py\_table\_of\_contents\_photo](/docs/opencv5/py_tutorials/py_photo/py_table_of_contents_photo/) - [Py Table Of Contents Setuppy\_tutorials/py\_setup/py\_table\_of\_contents\_setup](/docs/opencv5/py_tutorials/py_setup/py_table_of_contents_setup/) - [Py Table Of Contents Videopy\_tutorials/py\_video/py\_table\_of\_contents\_video](/docs/opencv5/py_tutorials/py_video/py_table_of_contents_video/) - [Animationstutorials/app/animations](/docs/opencv5/tutorials/app/animations/) - [Highgui Wayland Ubuntututorials/app/highgui\_wayland\_ubuntu](/docs/opencv5/tutorials/app/highgui_wayland_ubuntu/) - [Intelperctutorials/app/intelperc](/docs/opencv5/tutorials/app/intelperc/) - [Kinect Opennitutorials/app/kinect\_openni](/docs/opencv5/tutorials/app/kinect_openni/) - [Orbbec Astra Opennitutorials/app/orbbec\_astra\_openni](/docs/opencv5/tutorials/app/orbbec_astra_openni/) - [Orbbec Uvctutorials/app/orbbec\_uvc](/docs/opencv5/tutorials/app/orbbec_uvc/) - [Raster Io Gdaltutorials/app/raster\_io\_gdal](/docs/opencv5/tutorials/app/raster_io_gdal/) - [Table Of Content Apptutorials/app/table\_of\_content\_app](/docs/opencv5/tutorials/app/table_of_content_app/) - [Trackbartutorials/app/trackbar](/docs/opencv5/tutorials/app/trackbar/) - [Video Input Psnr Ssimtutorials/app/video\_input\_psnr\_ssim](/docs/opencv5/tutorials/app/video_input_psnr_ssim/) - [Video Writetutorials/app/video\_write](/docs/opencv5/tutorials/app/video_write/) - [Table Of Content Calib3dtutorials/calib3d/table\_of\_content\_calib3d](/docs/opencv5/tutorials/calib3d/table_of_content_calib3d/) - [Usactutorials/calib3d/usac](/docs/opencv5/tutorials/calib3d/usac/) - [Mat Operationstutorials/core/mat\_operations](/docs/opencv5/tutorials/core/mat_operations/) - [Table Of Content Coretutorials/core/table\_of\_content\_core](/docs/opencv5/tutorials/core/table_of_content_core/) - [Table Of Content Dnntutorials/dnn/table\_of\_content\_dnn](/docs/opencv5/tutorials/dnn/table_of_content_dnn/) - [Table Of Content Featurestutorials/features/table\_of\_content\_features](/docs/opencv5/tutorials/features/table_of_content_features/) - [Table Of Content Geometrytutorials/geometry/table\_of\_content\_geometry](/docs/opencv5/tutorials/geometry/table_of_content_geometry/) …and 256 more in the sidebar. [llms.txt](/docs/opencv5/llms.txt) for agents ## [Faq](https://docharvest.github.io/docs/opencv5/faq/) Contents opencv5 Faq OpenCV 5 Faq # Frequently Asked Questions {#faq} Compatibility page. FAQ migrated to the project [wiki](https://github.com/opencv/opencv/wiki/FAQ). ## [Js Basic Ops](https://docharvest.github.io/docs/opencv5/js_tutorials/js_core/js_basic_ops/js_basic_ops/) Contents opencv5 Js Basic Ops OpenCV 5 Js Basic Ops # Basic Operations on Images {#tutorial\_js\_basic\_ops} ## Goal - Learn how to access image properties - Learn how to construct Mat - Learn how to copy Mat - Learn how to convert the type of Mat - Learn how to use MatVector - Learn how to access pixel values and modify them - Learn how to set Region of Interest (ROI) - Learn how to split and merge images ## Accessing Image Properties Image properties include number of rows, columns and size, depth, channels, type of image data. @code{.js} let src = cv.imread("canvasInput"); console.log('image width: ' + src.cols + '\\n' + 'image height: ' + src.rows + '\\n' + 'image size: ' + src.size().width + '\*' + src.size().height + '\\n' + 'image depth: ' + src.depth() + '\\n' + 'image channels ' + src.channels() + '\\n' + 'image type: ' + src.type() + '\\n'); @endcode @note src.type() is very important while debugging because a large number of errors in OpenCV.js code are caused by invalid data type. ## How to construct Mat There are 4 basic constructors: @code{.js} // 1. default constructor let mat = new cv.Mat(); // 2. two-dimensional arrays by size and type let mat = new cv.Mat(size, type); // 3. two-dimensional arrays by rows, cols, and type let mat = new cv.Mat(rows, cols, type); // 4. two-dimensional arrays by rows, cols, and type with initialization value let mat = new cv.Mat(rows, cols, type, new cv.Scalar()); @endcode There are 3 static functions: @code{.js} // 1. Create a Mat which is full of zeros let mat = cv.Mat.zeros(rows, cols, type); // 2. Create a Mat which is full of ones let mat = cv.Mat.ones(rows, cols, type); // 3. Create a Mat which is an identity matrix let mat = cv.Mat.eye(rows, cols, type); @endcode There are 2 factory functions: @code{.js} // 1. Use JS array to construct a mat. // For example: let mat = cv.matFromArray(2, 2, cv.CV\_8UC1, \[1, 2, 3, 4\]); let mat = cv.matFromArray(rows, cols, type, array); // 2. Use imgData to construct a mat let ctx = canvas.getContext("2d"); let imgData = ctx.getImageData(0, 0, canvas.width, canvas.height); let mat = cv.matFromImageData(imgData); @endcode @note Don't forget to delete cv.Mat when you don't want to use it any more. ## How to copy Mat There are 2 ways to copy a Mat: @code{.js} // 1. Clone let dst = src.clone(); // 2. CopyTo(only entries indicated in the mask are copied) src.copyTo(dst, mask); @endcode ## How to convert the type of Mat We use the function: **convertTo(m, rtype, alpha = 1, beta = 0)** @param m output matrix; if it does not have a proper size or type before the operation, it is reallocated. @param rtype desired output matrix type or, rather, the depth since the number of channels are the same as the input has; if rtype is negative, the output matrix will have the same type as the input. @param alpha optional scale factor. @param beta optional delta added to the scaled values. @code{.js} src.convertTo(dst, rtype); @endcode ## How use MatVector @code{.js} let mat = new cv.Mat(); // Initialise a MatVector let matVec = new cv.MatVector(); // Push a Mat back into MatVector matVec.push\_back(mat); // Get a Mat fom MatVector let cnt = matVec.get(0); mat.delete(); matVec.delete(); cnt.delete(); @endcode @note Don't forget to delete cv.Mat, cv.MatVector and cnt(the Mat you get from MatVector) when you don't want to use them any more. ## Accessing and Modifying pixel values Firstly, you should know the following type relationship: Data Properties C++ Type JavaScript Typed Array Mat Type data uchar Uint8Array CV\_8U data8S char Int8Array CV\_8S data16U ushort Uint16Array CV\_16U data16S short Int16Array CV\_16S data32S int Int32Array CV\_32S data32F float Float32Array CV\_32F data64F double Float64Array CV\_64F **1\. data** @code{.js} let row = 3, col = 4; let src = cv.imread("canvasInput"); if (src.isContinuous()) { let R = src.data\[row \* src.cols \* src.channels() + col \* src.channels()\]; let G = src.data\[row \* src.cols \* src.channels() + col \* src.channels() + 1\]; let B = src.data\[row \* src.cols \* src.channels() + col \* src.channels() + 2\]; let A = src.data\[row \* src.cols \* src.channels() + col \* src.channels() + 3\]; } @endcode @note Data manipulation is only valid for continuous Mat. You should use isContinuous() to check first. **2\. at** Mat Type At Manipulation CV\_8U ucharAt CV\_8S charAt CV\_16U ushortAt CV\_16S shortAt CV\_32S intAt CV\_32F floatAt CV\_64F doubleAt @code{.js} let row = 3, col = 4; let src = cv.imread("canvasInput"); let R = src.ucharAt(row, col \* src.channels()); let G = src.ucharAt(row, col \* src.channels() + 1); let B = src.ucharAt(row, col \* src.channels() + 2); let A = src.ucharAt(row, col \* src.channels() + 3); @endcode @note At manipulation is only for single channel access and the value can't be modified. **3\. ptr** Mat Type Ptr Manipulation JavaScript Typed Array CV\_8U ucharPtr Uint8Array CV\_8S charPtr Int8Array CV\_16U ushortPtr Uint16Array CV\_16S shortPtr Int16Array CV\_32S intPtr Int32Array CV\_32F floatPtr Float32Array CV\_64F doublePtr Float64Array @code{.js} let row = 3, col = 4; let src = cv.imread("canvasInput"); let pixel = src.ucharPtr(row, col); let R = pixel\[0\]; let G = pixel\[1\]; let B = pixel\[2\]; let A = pixel\[3\]; @endcode mat.ucharPtr(k) get the k th row of the mat. mat.ucharPtr(i, j) get the i th row and the j th column of the mat. ## Image ROI Sometimes, you will have to play with certain region of images. For eye detection in images, first face detection is done all over the image and when face is obtained, we select the face region alone and search for eyes inside it instead of searching whole image. It improves accuracy (because eyes are always on faces) and performance (because we search for a small area) We use the function: **roi (rect)** @param rect rectangle Region of Interest. ## Try it \\htmlonly \\endhtmlonly ## Splitting and Merging Image Channels Sometimes you will need to work separately on R,G,B channels of image. Then you need to split the RGB images to single planes. Or another time, you may need to join these individual channels to RGB image. @code{.js} let src = cv.imread("canvasInput"); let rgbaPlanes = new cv.MatVector(); // Split the Mat cv.split(src, rgbaPlanes); // Get R channel let R = rgbaPlanes.get(0); // Merge all channels cv.merge(rgbaPlanes, src); src.delete(); rgbaPlanes.delete(); R.delete(); @endcode @note Don't forget to delete cv.Mat, cv.MatVector and R(the Mat you get from MatVector) when you don't want to use them any more. ## Making Borders for Images (Padding) If you want to create a border around the image, something like a photo frame, you can use **cv.copyMakeBorder()** function. But it has more applications for convolution operation, zero padding etc. This function takes following arguments: - **src** - input image - **top**, **bottom**, **left**, **right** - border width in number of pixels in corresponding directions - **borderType** - Flag defining what kind of border to be added. It can be following types: - **cv.BORDER\_CONSTANT** - Adds a constant colored border. The value should be given as next argument. - **cv.BORDER\_REFLECT** - Border will be mirror reflection of the border elements, like this : _fedcba|abcdefgh|hgfedcb_ - **cv.BORDER\_REFLECT\_101** or **cv.BORDER\_DEFAULT** - Same as above, but with a slight change, like this : _gfedcb|abcdefgh|gfedcba_ - **cv.BORDER\_REPLICATE** - Last element is replicated throughout, like this: _aaaaaa|abcdefgh|hhhhhhh_ - **cv.BORDER\_WRAP** - Can't explain, it will look like this : _cdefgh|abcdefgh|abcdefg_ - **value** - Color of border if border type is cv.BORDER\_CONSTANT ## Try it \\htmlonly \\endhtmlonly ## [Js Image Arithmetics](https://docharvest.github.io/docs/opencv5/js_tutorials/js_core/js_image_arithmetics/js_image_arithmetics/) Contents opencv5 Js Image Arithmetics OpenCV 5 Js Image Arithmetics # Arithmetic Operations on Images {#tutorial\_js\_image\_arithmetics} ## Goal - Learn several arithmetic operations on images like addition, subtraction, bitwise operations, etc. - You will learn these functions : **cv.add()**, **cv.subtract()**, etc. ## Image Addition You can add two images by OpenCV function, cv.add(). res = img1 + img2. Both images should be of same depth and type. For example, consider below sample: @code{.js} let src1 = cv.imread("canvasInput1"); let src2 = cv.imread("canvasInput2"); let dst = new cv.Mat(); let mask = new cv.Mat(); let dtype = -1; cv.add(src1, src2, dst, mask, dtype); src1.delete(); src2.delete(); dst.delete(); mask.delete(); @endcode ## Image Subtraction You can subtract two images by OpenCV function, cv.subtract(). res = img1 - img2. Both images should be of same depth and type. Note that when used with RGBA images, the alpha channel is also subtracted. For example, consider below sample: @code{.js} let src1 = cv.imread("canvasInput1"); let src2 = cv.imread("canvasInput2"); let dst = new cv.Mat(); let mask = new cv.Mat(); let dtype = -1; cv.subtract(src1, src2, dst, mask, dtype); src1.delete(); src2.delete(); dst.delete(); mask.delete(); @endcode ## Bitwise Operations This includes bitwise AND, OR, NOT and XOR operations. They will be highly useful while extracting any part of the image, defining and working with non-rectangular ROI etc. Below we will see an example on how to change a particular region of an image. I want to put OpenCV logo above an image. If I add two images, it will change color. If I blend it, I get an transparent effect. But I want it to be opaque. If it was a rectangular region, I could use ROI as we did in last chapter. But OpenCV logo is a not a rectangular shape. So you can do it with bitwise operations. ## Try it \\htmlonly \\endhtmlonly ## [Js Image Arithmetics](https://docharvest.github.io/docs/opencv5/js_tutorials/js_core/js_some_data_structures/js_image_arithmetics/) Contents opencv5 Js Image Arithmetics OpenCV 5 Js Image Arithmetics # Some Data Structures {#tutorial\_js\_some\_data\_structures} ## Goal - You will learn some data structures : **Point**, **Scalar**, **Size**, **Circle**, **Rect**, **RotatedRect** etc. Scalar is array type in Javascript. Point, Size, Circle, Rect and RotatedRect are object type in JavaScript. ## Point There are 2 ways to construct a Point and they are the same: @code{.js} // The first way let point = new cv.Point(x, y); // The second way let point = {x: x, y: y}; @endcode @param x x coordinate of the point.(the origin is the top left corner of the image) @param y y coordinate of the point. ## Scalar There are 2 ways to construct a Scalar and they are the same: @code{.js} // The first way let scalar = new cv.Scalar(R, G, B, Alpha); // The second way let scalar = \[R, G, B, Alpha\]; @endcode @param R pixel value of red channel. @param G pixel value of green channel. @param B pixel value of blue channel. @param Alpha pixel value of alpha channel. ## Size There are 2 ways to construct a Size and they are the same: @code{.js} // The first way let size = new cv.Size(width, height); // The second way let size = {width : width, height : height}; @endcode @param width the width of the size. @param height the height of the size. ## Circle There are 2 ways to construct a Circle and they are the same: @code{.js} // The first way let circle = new cv.Circle(center, radius); // The second way let circle = {center : center, radius : radius}; @endcode @param center the center of the circle. @param radius the radius of the circle. ## Rect There are 2 ways to construct a Rect and they are the same: @code{.js} // The first way let rect = new cv.Rect(x, y, width, height); // The second way let rect = {x : x, y : y, width : width, height : height}; @endcode @param x x coordinate of the vertex which is the top left corner of the rectangle. @param y y coordinate of the vertex which is the top left corner of the rectangle. @param width the width of the rectangle. @param height the height of the rectangle. ## RotatedRect There are 2 ways to construct a RotatedRect and they are the same: @code{.js} // The first way let rotatedRect = new cv.RotatedRect(center, size, angle); // The second way let rotatedRect = {center : center, size : size, angle : angle}; @endcode @param center the rectangle mass center. @param size width and height of the rectangle. @param angle the rotation angle in a clockwise direction. When the angle is 0, 90, 180, 270 etc., the rectangle becomes an up-right rectangle. Learn how to get the vertices from rotatedRect: We use the function: **cv.RotatedRect.points(rotatedRect)** @param rotatedRect rotated rectangle @code{.js} let vertices = cv.RotatedRect.points(rotatedRect); let point1 = vertices\[0\]; let point2 = vertices\[1\]; let point3 = vertices\[2\]; let point4 = vertices\[3\]; @endcode Learn how to get the bounding rectangle from rotatedRect: We use the function: **cv.RotatedRect.boundingRect(rotatedRect)** @param rotatedRect rotated rectangle @code{.js} let boundingRect = cv.RotatedRect.boundingRect(rotatedRect); @endcode ## [Js Table Of Contents Core](https://docharvest.github.io/docs/opencv5/js_tutorials/js_core/js_table_of_contents_core/) Contents opencv5 Js Table Of Contents Core OpenCV 5 Js Table Of Contents Core # Core Operations {#tutorial\_js\_table\_of\_contents\_core} - @subpage tutorial\_js\_basic\_ops Learn to read and edit pixel values, working with image ROI and other basic operations. - @subpage tutorial\_js\_image\_arithmetics Perform arithmetic operations on images - @subpage tutorial\_js\_some\_data\_structures Learn some data structures ## [Js Image Classification](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_image_classification/js_image_classification/) Contents opencv5 Js Image Classification OpenCV 5 Js Image Classification # Image Classification Example {#tutorial\_js\_image\_classification} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for image classification. \\htmlonly \\endhtmlonly ## [Js Image Classification With Camera](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_image_classification/js_image_classification_with_camera/) Contents opencv5 Js Image Classification With Camera OpenCV 5 Js Image Classification With Camera # Image Classification Example with Camera {#tutorial\_js\_image\_classification\_with\_camera} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for image classification example with camera. @note If you don't know how to capture video from camera, please review @ref tutorial\_js\_video\_display. \\htmlonly \\endhtmlonly ## [Js Object Detection](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_object_detection/js_object_detection/) Contents opencv5 Js Object Detection OpenCV 5 Js Object Detection # Object Detection Example {#tutorial\_js\_object\_detection} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for object detection. \\htmlonly \\endhtmlonly ## [Js Object Detection With Camera](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_object_detection/js_object_detection_with_camera/) Contents opencv5 Js Object Detection With Camera OpenCV 5 Js Object Detection With Camera # Object Detection Example with Camera{#tutorial\_js\_object\_detection\_with\_camera} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for object detection with camera. \\htmlonly \\endhtmlonly ## [Js Pose Estimation](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_pose_estimation/js_pose_estimation/) Contents opencv5 Js Pose Estimation OpenCV 5 Js Pose Estimation # Pose Estimation Example {#tutorial\_js\_pose\_estimation} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for pose estimation. \\htmlonly \\endhtmlonly ## [Js Semantic Segmentation](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_semantic_segmentation/js_semantic_segmentation/) Contents opencv5 Js Semantic Segmentation OpenCV 5 Js Semantic Segmentation # Semantic Segmentation Example {#tutorial\_js\_semantic\_segmentation} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for semantic segmentation. \\htmlonly \\endhtmlonly ## [Js Style Transfer](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_style_transfer/js_style_transfer/) Contents opencv5 Js Style Transfer OpenCV 5 Js Style Transfer # Style Transfer Example {#tutorial\_js\_style\_transfer} ## Goal - In this tutorial you will learn how to use OpenCV.js dnn module for style transfer. \\htmlonly \\endhtmlonly ## [Js Table Of Contents Dnn](https://docharvest.github.io/docs/opencv5/js_tutorials/js_dnn/js_table_of_contents_dnn/) Contents opencv5 Js Table Of Contents Dnn OpenCV 5 Js Table Of Contents Dnn # Deep Neural Networks (dnn module) {#tutorial\_js\_table\_of\_contents\_dnn} - @subpage tutorial\_js\_image\_classification Image classification example - @subpage tutorial\_js\_image\_classification\_with\_camera Image classification example with camera - @subpage tutorial\_js\_object\_detection Object detection example - @subpage tutorial\_js\_object\_detection\_with\_camera Object detection example with camera - @subpage tutorial\_js\_semantic\_segmentation Semantic segmentation example - @subpage tutorial\_js\_style\_transfer Style transfer example - @subpage tutorial\_js\_pose\_estimation Pose estimation example ## [Js Image Display](https://docharvest.github.io/docs/opencv5/js_tutorials/js_gui/js_image_display/js_image_display/) Contents opencv5 Js Image Display OpenCV 5 Js Image Display # Getting Started with Images {#tutorial\_js\_image\_display} ## Goals - Learn how to read an image and how to display it in a web. ## Read an image OpenCV.js saves images as cv.Mat type. We use HTML canvas element to transfer cv.Mat to the web or in reverse. The ImageData interface can represent or set the underlying pixel data of an area of a canvas element. @note Please refer to canvas docs for more details. First, create an ImageData obj from canvas: @code{.js} let canvas = document.getElementById(canvasInputId); let ctx = canvas.getContext('2d'); let imgData = ctx.getImageData(0, 0, canvas.width, canvas.height); @endcode Then, use cv.matFromImageData to construct a cv.Mat: @code{.js} let src = cv.matFromImageData(imgData); @endcode @note Because canvas only support 8-bit RGBA image with continuous storage, the cv.Mat type is cv.CV\_8UC4. It is different from native OpenCV because images returned and shown by the native **imread** and **imshow** have the channels stored in BGR order. ## Display an image First, convert the type of src to cv.CV\_8UC4: @code{.js} let dst = new cv.Mat(); // scale and shift are used to map the data to \[0, 255\]. src.convertTo(dst, cv.CV\_8U, scale, shift); // \*\*\* is GRAY, RGB, or RGBA, according to src.channels() is 1, 3 or 4. cv.cvtColor(dst, dst, cv.COLOR\_\*\*\*2RGBA); @endcode Then, new an ImageData obj from dst: @code{.js} let imgData = new ImageData(new Uint8ClampedArray(dst.data), dst.cols, dst.rows); @endcode Finally, display it: @code{.js} let canvas = document.getElementById(canvasOutputId); let ctx = canvas.getContext('2d'); ctx.clearRect(0, 0, canvas.width, canvas.height); canvas.width = imgData.width; canvas.height = imgData.height; ctx.putImageData(imgData, 0, 0); @endcode ## In OpenCV.js OpenCV.js implements image reading and showing using the above method. We use **cv.imread (imageSource)** to read an image from html canvas or img element. @param imageSource canvas element or id, or img element or id. @return mat with channels stored in RGBA order. We use **cv.imshow (canvasSource, mat)** to display it. The function may scale the mat, depending on its depth: - If the mat is 8-bit unsigned, it is displayed as is. - If the mat is 16-bit unsigned or 32-bit integer, the pixels are divided by 256. That is, the value range \[0,255\*256\] is mapped to \[0,255\]. - If the mat is 32-bit floating-point, the pixel values are multiplied by 255. That is, the value range \[0,1\] is mapped to \[0,255\]. @param canvasSource canvas element or id. @param mat mat to be shown. The above code of image reading and showing could be simplified as below. @code{.js} let img = cv.imread(imageSource); cv.imshow(canvasOutput, img); img.delete(); @endcode ## Try it \\htmlonly \\endhtmlonly ## [Js Table Of Contents Gui](https://docharvest.github.io/docs/opencv5/js_tutorials/js_gui/js_table_of_contents_gui/) Contents opencv5 Js Table Of Contents Gui OpenCV 5 Js Table Of Contents Gui # GUI Features {#tutorial\_js\_table\_of\_contents\_gui} - @subpage tutorial\_js\_image\_display Learn to load an image and display it in a web - @subpage tutorial\_js\_video\_display Learn to capture video from Camera and play it - @subpage tutorial\_js\_trackbar Create trackbar to control certain parameters ## [Js Trackbar](https://docharvest.github.io/docs/opencv5/js_tutorials/js_gui/js_trackbar/js_trackbar/) Contents opencv5 Js Trackbar OpenCV 5 Js Trackbar # Add a Trackbar to Your Application {#tutorial\_js\_trackbar} ## Goal - Use HTML DOM Input Range Object to add a trackbar to your application. ## Code Demo Here, we will create a simple application that blends two images. We will let the user enter the weight by using the trackbar. First, we need to create three canvas elements: two for input and one for output. Please refer to the tutorial @ref tutorial\_js\_image\_display. @code{.js} let src1 = cv.imread('canvasInput1'); let src2 = cv.imread('canvasInput2'); @endcode Then, we use HTML DOM Input Range Object to implement the trackbar, which is shown as below. @note elements with type="range" are not supported in Internet Explorer 9 and earlier versions. You can create an element with type="range" with the document.createElement() method: @code{.js} let x = document.createElement('INPUT'); x.setAttribute('type', 'range'); @endcode You can access an element with type="range" with getElementById(): @code{.js} let x = document.getElementById('myRange'); @endcode As a trackbar, the range element need a trackbar name, the default value, minimum value, maximum value, step and the callback function which is executed every time trackbar value changes. The callback function always has a default argument, which is the trackbar position. Additionally, a text element to display the trackbar value is fine. In our case, we can create the trackbar as below: @code{.html} Weight: @endcode Finally, we can use the trackbar value in the callback function, blend the two images, and display the result. @code{.js} let weightValue = document.getElementById('weightValue'); let trackbar = document.getElementById('trackbar'); weightValue.setAttribute('value', trackbar.value); let alpha = trackbar.value/trackbar.max; let beta = ( 1.0 - alpha ); let src1 = cv.imread('canvasInput1'); let src2 = cv.imread('canvasInput2'); let dst = new cv.Mat(); cv.addWeighted( src1, alpha, src2, beta, 0.0, dst, -1); cv.imshow('canvasOutput', dst); dst.delete(); src1.delete(); src2.delete(); @endcode @sa cv.addWeighted ## Try it \\htmlonly \\endhtmlonly ## [Js Video Display](https://docharvest.github.io/docs/opencv5/js_tutorials/js_gui/js_video_display/js_video_display/) ## [Js Canny](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_canny/js_canny/) Contents opencv5 Js Canny OpenCV 5 Js Canny # Canny Edge Detection {#tutorial\_js\_canny} ## Goal - Concept of Canny edge detection - OpenCV functions for that : **cv.Canny()** ## Theory Canny Edge Detection is a popular edge detection algorithm. It was developed by John F. Canny in 1986. It is a multi-stage algorithm and we will go through each stages. \-# **Noise Reduction** ``` Since edge detection is susceptible to noise in the image, first step is to remove the noise in the image with a 5x5 Gaussian filter. We have already seen this in previous chapters. ``` \-# **Finding Intensity Gradient of the Image** ``` Smoothened image is then filtered with a Sobel kernel in both horizontal and vertical direction to get first derivative in horizontal direction (\f$G_x\f$) and vertical direction (\f$G_y\f$). From these two images, we can find edge gradient and direction for each pixel as follows: \f[ Edge\_Gradient \; (G) = \sqrt{G_x^2 + G_y^2} \\ Angle \; (\theta) = \tan^{-1} \bigg(\frac{G_y}{G_x}\bigg) \f] Gradient direction is always perpendicular to edges. It is rounded to one of four angles representing vertical, horizontal and two diagonal directions. ``` \-# **Non-maximum Suppression** ``` After getting gradient magnitude and direction, a full scan of image is done to remove any unwanted pixels which may not constitute the edge. For this, at every pixel, pixel is checked if it is a local maximum in its neighborhood in the direction of gradient. Check the image below: ![image](images/nms.jpg) Point A is on the edge ( in vertical direction). Gradient direction is normal to the edge. Point B and C are in gradient directions. So point A is checked with point B and C to see if it forms a local maximum. If so, it is considered for next stage, otherwise, it is suppressed ( put to zero). In short, the result you get is a binary image with "thin edges". ``` \-# **Hysteresis Thresholding** ``` This stage decides which are all edges are really edges and which are not. For this, we need two threshold values, minVal and maxVal. Any edges with intensity gradient more than maxVal are sure to be edges and those below minVal are sure to be non-edges, so discarded. Those who lie between these two thresholds are classified edges or non-edges based on their connectivity. If they are connected to "sure-edge" pixels, they are considered to be part of edges. Otherwise, they are also discarded. See the image below: ![image](images/hysteresis.jpg) The edge A is above the maxVal, so considered as "sure-edge". Although edge C is below maxVal, it is connected to edge A, so that also considered as valid edge and we get that full curve. But edge B, although it is above minVal and is in same region as that of edge C, it is not connected to any "sure-edge", so that is discarded. So it is very important that we have to select minVal and maxVal accordingly to get the correct result. This stage also removes small pixels noises on the assumption that edges are long lines. ``` So what we finally get is strong edges in the image. ## Canny Edge Detection in OpenCV We use the function: **cv.Canny(image, edges, threshold1, threshold2, apertureSize = 3, L2gradient = false)** @param image 8-bit input image. @param edges output edge map; single channels 8-bit image, which has the same size as image. @param threshold1 first threshold for the hysteresis procedure. @param threshold2 second threshold for the hysteresis procedure.. @param apertureSize aperture size for the Sobel operator. @param L2gradient specifies the equation for finding gradient magnitude. If it is True, it uses the equation mentioned above which is more accurate, otherwise it uses this function: \\f$Edge\_Gradient ; (G) = |G\_x| + |G\_y|\\f$. ## Try it \\htmlonly \\endhtmlonly ## [Js Colorspaces](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_colorspaces/js_colorspaces/) Contents opencv5 Js Colorspaces OpenCV 5 Js Colorspaces # Changing Colorspaces {#tutorial\_js\_colorspaces} ## Goal - In this tutorial, you will learn how to convert images from one color-space to another, like RGB \\f$\\leftrightarrow\\f$ Gray, RGB \\f$\\leftrightarrow\\f$ HSV etc. - You will learn following functions : **cv.cvtColor()**, **cv.inRange()** etc. ## cvtColor There are more than 150 color-space conversion methods available in OpenCV. But we will look into the most widely used one: RGB \\f$\\leftrightarrow\\f$ Gray. We use the function: **cv.cvtColor (src, dst, code, dstCn = 0)** @param src input image. @param dst output image of the same size and depth as src @param code color space conversion code(see **cv.ColorConversionCodes**). @param dstCn number of channels in the destination image; if the parameter is 0, the number of the channels is derived automatically from src and code. For RGB \\f$\\rightarrow\\f$ Gray conversion we use the code cv.COLOR\_RGBA2GRAY. ## Try it \\htmlonly \\endhtmlonly ## inRange Checks if array elements lie between the elements of two other arrays. We use the function: **cv.inRange (src, lowerb, upperb, dst)** @param src first input image. @param lowerb inclusive lower boundary Mat of the same size as src. @param upperb inclusive upper boundary Mat of the same size as src. @param dst output image of the same size as src and cv.CV\_8U type. ## Try it \\htmlonly \\endhtmlonly ## [Js Contour Features](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_contour_features/js_contour_features/) Contents opencv5 Js Contour Features OpenCV 5 Js Contour Features # Contour Features {#tutorial\_js\_contour\_features} @prev\_tutorial{tutorial\_js\_contours\_begin} @next\_tutorial{tutorial\_js\_contour\_properties} ## Goal - To find the different features of contours, like area, perimeter, centroid, bounding box etc - You will learn plenty of functions related to contours. 1. Moments * * * Image moments help you to calculate some features like center of mass of the object, area of the object etc. Check out the wikipedia page on [Image Moments](http://en.wikipedia.org/wiki/Image_moment) We use the function: **cv.moments (array, binaryImage = false)** @param array raster image (single-channel, 8-bit or floating-point 2D array) or an array ( 1×N or N×1 ) of 2D points. @param binaryImage if it is true, all non-zero image pixels are treated as 1's. The parameter is used for images only. ## Try it \\htmlonly \\endhtmlonly From this moments, you can extract useful data like area, centroid etc. Centroid is given by the relations, \\f$C\_x = \\frac{M\_{10}}{M\_{00}}\\f$ and \\f$C\_y = \\frac{M\_{01}}{M\_{00}}\\f$. This can be done as follows: @code{.js} let cx = M.m10/M.m00 let cy = M.m01/M.m00 @endcode 2. Contour Area * * * Contour area is given by the function **cv.contourArea()** or from moments, **M\['m00'\]**. We use the function: **cv.contourArea (contour, oriented = false)** @param contour input vector of 2D points (contour vertices) @param oriented oriented area flag. If it is true, the function returns a signed area value, depending on the contour orientation (clockwise or counter-clockwise). Using this feature you can determine orientation of a contour by taking the sign of an area. By default, the parameter is false, which means that the absolute value is returned. ## Try it \\htmlonly \\endhtmlonly 3. Contour Perimeter * * * It is also called arc length. It can be found out using **cv.arcLength()** function. We use the function: **cv.arcLength (curve, closed)** @param curve input vector of 2D points. @param closed flag indicating whether the curve is closed or not. ## Try it \\htmlonly \\endhtmlonly 4. Contour Approximation * * * It approximates a contour shape to another shape with less number of vertices depending upon the precision we specify. It is an implementation of [Douglas-Peucker algorithm](http://en.wikipedia.org/wiki/Ramer-Douglas-Peucker_algorithm). Check the wikipedia page for algorithm and demonstration. We use the function: **cv.approxPolyDP (curve, approxCurve, epsilon, closed)** @param curve input vector of 2D points stored in cv.Mat. @param approxCurve result of the approximation. The type should match the type of the input curve. @param epsilon parameter specifying the approximation accuracy. This is the maximum distance between the original curve and its approximation. @param closed If true, the approximated curve is closed (its first and last vertices are connected). Otherwise, it is not closed. ## Try it \\htmlonly \\endhtmlonly 5. Convex Hull * * * Convex Hull will look similar to contour approximation, but it is not (Both may provide same results in some cases). Here, **cv.convexHull()** function checks a curve for convexity defects and corrects it. Generally speaking, convex curves are the curves which are always bulged out, or at-least flat. And if it is bulged inside, it is called convexity defects. For example, check the below image of hand. Red line shows the convex hull of hand. The double-sided arrow marks shows the convexity defects, which are the local maximum deviations of hull from contours. We use the function: **cv.convexHull (points, hull, clockwise = false, returnPoints = true)** @param points input 2D point set. @param hull output convex hull. @param clockwise orientation flag. If it is true, the output convex hull is oriented clockwise. Otherwise, it is oriented counter-clockwise. The assumed coordinate system has its X axis pointing to the right, and its Y axis pointing upwards. @param returnPoints operation flag. In case of a matrix, when the flag is true, the function returns convex hull points. Otherwise, it returns indices of the convex hull points. ## Try it \\htmlonly \\endhtmlonly 6. Checking Convexity * * * There is a function to check if a curve is convex or not, **cv.isContourConvex()**. It just return whether True or False. Not a big deal. @code{.js} cv.isContourConvex(cnt); @endcode 7. Bounding Rectangle * * * There are two types of bounding rectangles. ### 7.a. Straight Bounding Rectangle It is a straight rectangle, it doesn't consider the rotation of the object. So area of the bounding rectangle won't be minimum. We use the function: **cv.boundingRect (points)** @param points input 2D point set. ## Try it \\htmlonly \\endhtmlonly ### 7.b. Rotated Rectangle Here, bounding rectangle is drawn with minimum area, so it considers the rotation also. We use the function: **cv.minAreaRect (points)** @param points input 2D point set. ## Try it \\htmlonly \\endhtmlonly 8. Minimum Enclosing Circle * * * Next we find the circumcircle of an object using the function **cv.minEnclosingCircle()**. It is a circle which completely covers the object with minimum area. We use the functions: **cv.minEnclosingCircle (points)** @param points input 2D point set. **cv.circle (img, center, radius, color, thickness = 1, lineType = cv.LINE\_8, shift = 0)** @param img image where the circle is drawn. @param center center of the circle. @param radius radius of the circle. @param color circle color. @param thickness thickness of the circle outline, if positive. Negative thickness means that a filled circle is to be drawn. @param lineType type of the circle boundary. @param shift number of fractional bits in the coordinates of the center and in the radius value. ## Try it \\htmlonly \\endhtmlonly 9. Fitting an Ellipse * * * Next one is to fit an ellipse to an object. It returns the rotated rectangle in which the ellipse is inscribed. We use the functions: **cv.fitEllipse (points)** @param points input 2D point set. **cv.ellipse1 (img, box, color, thickness = 1, lineType = cv.LINE\_8)** @param img image. @param box alternative ellipse representation via RotatedRect. This means that the function draws an ellipse inscribed in the rotated rectangle. @param color ellipse color. @param thickness thickness of the ellipse arc outline, if positive. Otherwise, this indicates that a filled ellipse sector is to be drawn. @param lineType type of the ellipse boundary. ## Try it \\htmlonly \\endhtmlonly 10. Fitting a Line * * * Similarly we can fit a line to a set of points. We can approximate a straight line to it. We use the functions: **cv.fitLine (points, line, distType, param, reps, aeps)** @param points input 2D point set. @param line output line parameters. It should be a Mat of 4 elements\[vx, vy, x0, y0\], where \[vx, vy\] is a normalized vector collinear to the line and \[x0, y0\] is a point on the line. @param distType distance used by the M-estimator(see cv.DistanceTypes). @param param numerical parameter ( C ) for some types of distances. If it is 0, an optimal value is chosen. @param reps sufficient accuracy for the radius (distance between the coordinate origin and the line). @param aeps sufficient accuracy for the angle. 0.01 would be a good default value for reps and aeps. **cv.line (img, pt1, pt2, color, thickness = 1, lineType = cv.LINE\_8, shift = 0)** @param img image. @param pt1 first point of the line segment. @param pt2 second point of the line segment. @param color line color. @param thickness line thickness. @param lineType type of the line,. @param shift number of fractional bits in the point coordinates. ## Try it \\htmlonly \\endhtmlonly ## [Js Contour Properties](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_contour_properties/js_contour_properties/) Contents opencv5 Js Contour Properties OpenCV 5 Js Contour Properties # Contour Properties {#tutorial\_js\_contour\_properties} @prev\_tutorial{tutorial\_js\_contour\_features} @next\_tutorial{tutorial\_js\_contours\_more\_functions} ## Goal - Here we will learn to extract some frequently used properties of objects like Solidity, Equivalent Diameter, Mask image, Mean Intensity etc. 1. Aspect Ratio * * * It is the ratio of width to height of bounding rect of the object. \\f\[Aspect ; Ratio = \\frac{Width}{Height}\\f\] @code{.js} let rect = cv.boundingRect(cnt); let aspectRatio = rect.width / rect.height; @endcode 2. Extent * * * Extent is the ratio of contour area to bounding rectangle area. \\f\[Extent = \\frac{Object ; Area}{Bounding ; Rectangle ; Area}\\f\] @code{.js} let area = cv.contourArea(cnt, false); let rect = cv.boundingRect(cnt)); let rectArea = rect.width \* rect.height; let extent = area / rectArea; @endcode 3. Solidity * * * Solidity is the ratio of contour area to its convex hull area. \\f\[Solidity = \\frac{Contour ; Area}{Convex ; Hull ; Area}\\f\] @code{.js} let area = cv.contourArea(cnt, false); cv.convexHull(cnt, hull, false, true); let hullArea = cv.contourArea(hull, false); let solidity = area / hullArea; @endcode 4. Equivalent Diameter * * * Equivalent Diameter is the diameter of the circle whose area is same as the contour area. \\f\[Equivalent ; Diameter = \\sqrt{\\frac{4 \\times Contour ; Area}{\\pi}}\\f\] @code{.js} let area = cv.contourArea(cnt, false); let equiDiameter = Math.sqrt(4 \* area / Math.PI); @endcode 5. Orientation * * * Orientation is the angle at which object is directed. Following method also gives the Major Axis and Minor Axis lengths. @code{.js} let rotatedRect = cv.fitEllipse(cnt); let angle = rotatedRect.angle; @endcode 6. Mask and Pixel Points * * * In some cases, we may need all the points which comprises that object. We use the function: **cv.transpose (src, dst)** @param src input array. @param dst output array of the same type as src. \\htmlonly \\endhtmlonly 7. Maximum Value, Minimum Value and their locations * * * We use the function: **cv.minMaxLoc(src, mask)** @param src input single-channel array. @param mask optional mask used to select a sub-array. @code{.js} let result = cv.minMaxLoc(src, mask); let minVal = result.minVal; let maxVal = result.maxVal; let minLoc = result.minLoc; let maxLoc = result.maxLoc; @endcode 8. Mean Color or Mean Intensity * * * Here, we can find the average color of an object. Or it can be average intensity of the object in grayscale mode. We again use the same mask to do it. We use the function: **cv.mean (src, mask)** @param src input array that should have from 1 to 4 channels so that the result can be stored in Scalar. @param mask optional operation mask. @code{.js} let average = cv.mean(src, mask); @endcode ## [Js Contours Begin](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_contours_begin/js_contours_begin/) Contents opencv5 Js Contours Begin OpenCV 5 Js Contours Begin # Contours : Getting Started {#tutorial\_js\_contours\_begin} @next\_tutorial{tutorial\_js\_contour\_features} ## Goal - Understand what contours are. - Learn to find contours, draw contours etc - You will learn these functions : **cv.findContours()**, **cv.drawContours()** ## What are contours? Contours can be explained simply as a curve joining all the continuous points (along the boundary), having same color or intensity. The contours are a useful tool for shape analysis and object detection and recognition. - For better accuracy, use binary images. So before finding contours, apply threshold or canny edge detection. - Since opencv 3.2 source image is not modified by this function. - In OpenCV, finding contours is like finding white object from black background. So remember, object to be found should be white and background should be black. ## How to draw the contours? To draw the contours, cv.drawContours function is used. It can also be used to draw any shape provided you have its boundary points. We use the functions: **cv.findContours (image, contours, hierarchy, mode, method, offset = new cv.Point(0, 0))** @param image source, an 8-bit single-channel image. Non-zero pixels are treated as 1's. Zero pixels remain 0's, so the image is treated as binary. @param contours detected contours. @param hierarchy containing information about the image topology. It has as many elements as the number of contours. @param mode contour retrieval mode(see cv.RetrievalModes). @param method contour approximation method(see cv.ContourApproximationModes). @param offset optional offset by which every contour point is shifted. This is useful if the contours are extracted from the image ROI and then they should be analyzed in the whole image context. **cv.drawContours (image, contours, contourIdx, color, thickness = 1, lineType = cv.LINE\_8, hierarchy = new cv.Mat(), maxLevel = INT\_MAX, offset = new cv.Point(0, 0))** @param image destination image. @param contours all the input contours. @param contourIdx parameter indicating a contour to draw. If it is negative, all the contours are drawn. @param color color of the contours. @param thickness thickness of lines the contours are drawn with. If it is negative, the contour interiors are drawn. @param lineType line connectivity(see cv.LineTypes). @param hierarchy optional information about hierarchy. It is only needed if you want to draw only some of the contours(see maxLevel). @param maxLevel maximal level for drawn contours. If it is 0, only the specified contour is drawn. If it is 1, the function draws the contour(s) and all the nested contours. If it is 2, the function draws the contours, all the nested contours, all the nested-to-nested contours, and so on. This parameter is only taken into account when there is hierarchy available. @param offset optional contour shift parameter. ## Try it \\htmlonly \\endhtmlonly # Contour Approximation Method This is the fifth argument in cv.findContours function. What does it denote actually? Above, we told that contours are the boundaries of a shape with same intensity. It stores the (x,y) coordinates of the boundary of a shape. But does it store all the coordinates ? That is specified by this contour approximation method. If you pass cv.ContourApproximationModes.CHAIN\_APPROX\_NONE.value, all the boundary points are stored. But actually do we need all the points? For eg, you found the contour of a straight line. Do you need all the points on the line to represent that line? No, we need just two end points of that line. This is what cv.CHAIN\_APPROX\_SIMPLE does. It removes all redundant points and compresses the contour, thereby saving memory. ## [Js Contours Hierarchy](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_contours_hierarchy/js_contours_hierarchy/) Contents opencv5 Js Contours Hierarchy OpenCV 5 Js Contours Hierarchy # Contours Hierarchy {#tutorial\_js\_contours\_hierarchy} @prev\_tutorial{tutorial\_js\_contours\_more\_functions} ## Goal - This time, we learn about the hierarchy of contours, i.e. the parent-child relationship in Contours. ## Theory In the last few articles on contours, we have worked with several functions related to contours provided by OpenCV. But when we found the contours in image using **cv.findContours()** function, we have passed an argument, **Contour Retrieval Mode**. We usually passed **cv.RETR\_LIST** or **cv.RETR\_TREE** and it worked nice. But what does it actually mean ? Also, in the output, we got three arrays, first is the image, second is our contours, and one more output which we named as **hierarchy** (Please checkout the codes in previous articles). But we never used this hierarchy anywhere. Then what is this hierarchy and what is it for ? What is its relationship with the previous mentioned function argument ? That is what we are going to deal in this article. ### What is Hierarchy? Normally we use the **cv.findContours()** function to detect objects in an image, right ? Sometimes objects are in different locations. But in some cases, some shapes are inside other shapes. Just like nested figures. In this case, we call outer one as **parent** and inner one as **child**. This way, contours in an image has some relationship to each other. And we can specify how one contour is connected to each other, like, is it child of some other contour, or is it a parent etc. Representation of this relationship is called the **Hierarchy**. Consider an example image below : In this image, there are a few shapes which I have numbered from **0-5**. _2 and 2a_ denotes the external and internal contours of the outermost box. Here, contours 0,1,2 are **external or outermost**. We can say, they are in **hierarchy-0** or simply they are in **same hierarchy level**. Next comes **contour-2a**. It can be considered as a **child of contour-2** (or in opposite way, contour-2 is parent of contour-2a). So let it be in **hierarchy-1**. Similarly contour-3 is child of contour-2a and it comes in next hierarchy. Finally contours 4,5 are the children of contour-3a, and they come in the last hierarchy level. From the way I numbered the boxes, I would say contour-4 is the first child of contour-3a (It can be contour-5 also). I mentioned these things to understand terms like **same hierarchy level**, **external contour**, **child contour**, **parent contour**, **first child** etc. Now let's get into OpenCV. ### Hierarchy Representation in OpenCV So each contour has its own information regarding what hierarchy it is, who is its child, who is its parent etc. OpenCV represents it as an array of four values : **\[Next, Previous, First\_Child, Parent\]** \*"Next denotes next contour at the same hierarchical level."\* For eg, take contour-0 in our picture. Who is next contour in its same level ? It is contour-1. So simply put Next = 1. Similarly for Contour-1, next is contour-2. So Next = 2. What about contour-2? There is no next contour in the same level. So simply, put Next = -1. What about contour-4? It is in same level with contour-5. So its next contour is contour-5, so Next = 5. \*"Previous denotes previous contour at the same hierarchical level."\* It is same as above. Previous contour of contour-1 is contour-0 in the same level. Similarly for contour-2, it is contour-1. And for contour-0, there is no previous, so put it as -1. \*"First\_Child denotes its first child contour."\* There is no need of any explanation. For contour-2, child is contour-2a. So it gets the corresponding index value of contour-2a. What about contour-3a? It has two children. But we take only first child. And it is contour-4. So First\_Child = 4 for contour-3a. \*"Parent denotes index of its parent contour."\* It is just opposite of **First\_Child**. Both for contour-4 and contour-5, parent contour is contour-3a. For contour-3a, it is contour-3 and so on. @note If there is no child or parent, that field is taken as -1 So now we know about the hierarchy style used in OpenCV, we can check into Contour Retrieval Modes in OpenCV with the help of same image given above. ie what do flags like cv.RETR\_LIST, cv.RETR\_TREE, cv.RETR\_CCOMP, cv.RETR\_EXTERNAL etc mean? ## Contour Retrieval Mode ### 1\. RETR\_LIST This is the simplest of the four flags (from explanation point of view). It simply retrieves all the contours, but doesn't create any parent-child relationship. **Parents and kids are equal under this rule, and they are just contours**. ie they all belongs to same hierarchy level. So here, 3rd and 4th term in hierarchy array is always -1. But obviously, Next and Previous terms will have their corresponding values. ### 2\. RETR\_EXTERNAL If you use this flag, it returns only extreme outer flags. All child contours are left behind. **We can say, under this law, Only the eldest in every family is taken care of. It doesn't care about other members of the family)**. ### 3\. RETR\_CCOMP This flag retrieves all the contours and arranges them to a 2-level hierarchy. ie external contours of the object (ie its boundary) are placed in hierarchy-1. And the contours of holes inside object (if any) is placed in hierarchy-2. If any object inside it, its contour is placed again in hierarchy-1 only. And its hole in hierarchy-2 and so on. Just consider the image of a "big white zero" on a black background. Outer circle of zero belongs to first hierarchy, and inner circle of zero belongs to second hierarchy. We can explain it with a simple image. Here I have labelled the order of contours in red color and the hierarchy they belongs to, in green color (either 1 or 2). The order is same as the order OpenCV detects contours. So consider first contour, ie contour-0. It is hierarchy-1. It has two holes, contours 1&2, and they belong to hierarchy-2. So for contour-0, Next contour in same hierarchy level is contour-3. And there is no previous one. And its first is child is contour-1 in hierarchy-2. It has no parent, because it is in hierarchy-1. So its hierarchy array is \[3,-1,1,-1\] Now take contour-1. It is in hierarchy-2. Next one in same hierarchy (under the parenthood of contour-1) is contour-2. No previous one. No child, but parent is contour-0. So array is \[2,-1,-1,0\]. Similarly contour-2 : It is in hierarchy-2. There is not next contour in same hierarchy under contour-0. So no Next. Previous is contour-1. No child, parent is contour-0. So array is \[-1,1,-1,0\]. Contour - 3 : Next in hierarchy-1 is contour-5. Previous is contour-0. Child is contour-4 and no parent. So array is \[5,0,4,-1\]. Contour - 4 : It is in hierarchy 2 under contour-3 and it has no sibling. So no next, no previous, no child, parent is contour-3. So array is \[-1,-1,-1,3\]. ### 4\. RETR\_TREE And this is the final guy, Mr.Perfect. It retrieves all the contours and creates a full family hierarchy list. **It even tells, who is the grandpa, father, son, grandson and even beyond... :)**. For example, I took above image, rewrite the code for cv.RETR\_TREE, reorder the contours as per the result given by OpenCV and analyze it. Again, red letters give the contour number and green letters give the hierarchy order. Take contour-0 : It is in hierarchy-0. Next contour in same hierarchy is contour-7. No previous contours. Child is contour-1. And no parent. So array is \[7,-1,1,-1\]. Take contour-2 : It is in hierarchy-1. No contour in same level. No previous one. Child is contour-2. Parent is contour-0. So array is \[-1,-1,2,0\]. ## [Js Contours More Functions](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_contours_more_functions/js_contours_more_functions/) Contents opencv5 Js Contours More Functions OpenCV 5 Js Contours More Functions # Contours : More Functions {#tutorial\_js\_contours\_more\_functions} @prev\_tutorial{tutorial\_js\_contour\_properties} @next\_tutorial{tutorial\_js\_contours\_hierarchy} ## Goal - Convexity defects and how to find them. - Finding shortest distance from a point to a polygon - Matching different shapes ## Theory and Code ### 1\. Convexity Defects We saw what is convex hull in second chapter about contours. Any deviation of the object from this hull can be considered as convexity defect.We can visualize it using an image. We draw a line joining start point and end point, then draw a circle at the farthest point. @note Remember we have to pass returnPoints = False while finding convex hull, in order to find convexity defects. We use the function: **cv.convexityDefects (contour, convexhull, convexityDefect)** @param contour input contour. @param convexhull convex hull obtained using convexHull that should contain indices of the contour points that make the hull @param convexityDefect the output vector of convexity defects. Each convexity defect is represented as 4-element(start\_index, end\_index, farthest\_pt\_index, fixpt\_depth), where indices are 0-based indices in the original contour of the convexity defect beginning, end and the farthest point, and fixpt\_depth is fixed-point approximation (with 8 fractional bits) of the distance between the farthest contour point and the hull. That is, to get the floating-point value of the depth will be fixpt\_depth/256.0. ## Try it \\htmlonly \\endhtmlonly ### 2\. Point Polygon Test This function finds the shortest distance between a point in the image and a contour. It returns the distance which is negative when point is outside the contour, positive when point is inside and zero if point is on the contour. We use the function: **cv.pointPolygonTest (contour, pt, measureDist)** @param contour input contour. @param pt point tested against the contour. @param measureDist if true, the function estimates the signed distance from the point to the nearest contour edge. Otherwise, the function only checks if the point is inside a contour or not. @code{.js} let dist = cv.pointPolygonTest(cnt, new cv.Point(50, 50), true); @endcode ### 3\. Match Shapes OpenCV comes with a function **cv.matchShapes()** which enables us to compare two shapes, or two contours and returns a metric showing the similarity. The lower the result, the better match it is. It is calculated based on the hu-moment values. Different measurement methods are explained in the docs. We use the function: **cv.matchShapes (contour1, contour2, method, parameter)** @param contour1 first contour or grayscale image. @param contour2 second contour or grayscale image. @param method comparison method, see cv::ShapeMatchModes @param parameter method-specific parameter(not supported now). ## Try it \\htmlonly \\endhtmlonly ## [Js Table Of Contents Contours](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_contours/js_table_of_contents_contours/) Contents opencv5 Js Table Of Contents Contours OpenCV 5 Js Table Of Contents Contours # Contours in OpenCV.js {#tutorial\_js\_table\_of\_contents\_contours} - @subpage tutorial\_js\_contours\_begin Learn to find and draw Contours. - @subpage tutorial\_js\_contour\_features Learn to find different features of contours like area, perimeter, bounding rectangle etc. - @subpage tutorial\_js\_contour\_properties Learn to find different properties of contours like Solidity, Mean Intensity etc. - @subpage tutorial\_js\_contours\_more\_functions Learn to find convexity defects, pointPolygonTest, match different shapes etc. - @subpage tutorial\_js\_contours\_hierarchy Learn about Contour Hierarchy ## [Js Filtering](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_filtering/js_filtering/) Contents opencv5 Js Filtering OpenCV 5 Js Filtering # Smoothing Images {#tutorial\_js\_filtering} ## Goals - Blur the images with various low pass filters - Apply custom-made filters to images (2D convolution) ## 2D Convolution ( Image Filtering ) As in one-dimensional signals, images also can be filtered with various low-pass filters(LPF), high-pass filters(HPF) etc. LPF helps in removing noises, blurring the images etc. HPF filters helps in finding edges in the images. OpenCV provides a function **cv.filter2D()** to convolve a kernel with an image. As an example, we will try an averaging filter on an image. A 5x5 averaging filter kernel will look like below: \\f\[K = \\frac{1}{25} \\begin{bmatrix} 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 & 1 \\end{bmatrix}\\f\] We use the functions: **cv.filter2D (src, dst, ddepth, kernel, anchor = new cv.Point(-1, -1), delta = 0, borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image of the same size and the same number of channels as src. @param ddepth desired depth of the destination image. @param kernel convolution kernel (or rather a correlation kernel), a single-channel floating point matrix; if you want to apply different kernels to different channels, split the image into separate color planes using split and process them individually. @param anchor anchor of the kernel that indicates the relative position of a filtered point within the kernel; the anchor should lie within the kernel; default value new cv.Point(-1, -1) means that the anchor is at the kernel center. @param delta optional value added to the filtered pixels before storing them in dst. @param borderType pixel extrapolation method(see cv.BorderTypes). ## Try it \\htmlonly \\endhtmlonly ## Image Blurring (Image Smoothing) Image blurring is achieved by convolving the image with a low-pass filter kernel. It is useful for removing noises. It actually removes high frequency content (eg: noise, edges) from the image. So edges are blurred a little bit in this operation. (Well, there are blurring techniques which doesn't blur the edges too). OpenCV provides mainly four types of blurring techniques. ### 1\. Averaging This is done by convolving image with a normalized box filter. It simply takes the average of all the pixels under kernel area and replace the central element. This is done by the function **cv.blur()** or **cv.boxFilter()**. Check the docs for more details about the kernel. We should specify the width and height of kernel. A 3x3 normalized box filter would look like below: \\f\[K = \\frac{1}{9} \\begin{bmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \\end{bmatrix}\\f\] We use the functions: **cv.blur (src, dst, ksize, anchor = new cv.Point(-1, -1), borderType = cv.BORDER\_DEFAULT)** @param src input image; it can have any number of channels, which are processed independently, but the depth should be CV\_8U, CV\_16U, CV\_16S, CV\_32F or CV\_64F. @param dst output image of the same size and type as src. @param ksize blurring kernel size. @param anchor anchor point; anchor = new cv.Point(-1, -1) means that the anchor is at the kernel center. @param borderType border mode used to extrapolate pixels outside of the image(see cv.BorderTypes). **cv.boxFilter (src, dst, ddepth, ksize, anchor = new cv.Point(-1, -1), normalize = true, borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image of the same size and type as src. @param ddepth the output image depth (-1 to use src.depth()). @param ksize blurring kernel size. @param anchor anchor point; anchor = new cv.Point(-1, -1) means that the anchor is at the kernel center. @param normalize flag, specifying whether the kernel is normalized by its area or not. @param borderType border mode used to extrapolate pixels outside of the image(see cv.BorderTypes). @note If you don't want to use normalized box filter, use **cv.boxFilter()**. Pass an argument normalize = false to the function. ## Try it \\htmlonly \\endhtmlonly ### 2\. Gaussian Blurring In this, instead of box filter, gaussian kernel is used. We use the function: **cv.GaussianBlur (src, dst, ksize, sigmaX, sigmaY = 0, borderType = cv.BORDER\_DEFAULT)** @param src input image; the image can have any number of channels, which are processed independently, but the depth should be CV\_8U, CV\_16U, CV\_16S, CV\_32F or CV\_64F. @param dst output image of the same size and type as src. @param ksize blurring kernel size. @param sigmaX Gaussian kernel standard deviation in X direction. @param sigmaY Gaussian kernel standard deviation in Y direction; if sigmaY is zero, it is set to be equal to sigmaX, if both sigmas are zeros, they are computed from ksize.width and ksize.height, to fully control the result regardless of possible future modifications of all this semantics, it is recommended to specify all of ksize, sigmaX, and sigmaY. @param borderType pixel extrapolation method(see cv.BorderTypes). ## Try it \\htmlonly \\endhtmlonly ### 3\. Median Blurring Here, the function **cv.medianBlur()** takes median of all the pixels under kernel area and central element is replaced with this median value. This is highly effective against salt-and-pepper noise in the images. Interesting thing is that, in the above filters, central element is a newly calculated value which may be a pixel value in the image or a new value. But in median blurring, central element is always replaced by some pixel value in the image. It reduces the noise effectively. Its kernel size should be a positive odd integer. We use the function: **cv.medianBlur (src, dst, ksize)** @param src input 1, 3, or 4 channel image; when ksize is 3 or 5, the image depth should be cv.CV\_8U, cv.CV\_16U, or cv.CV\_32F, for larger aperture sizes, it can only be cv.CV\_8U. @param dst destination array of the same size and type as src. @param ksize aperture linear size; it must be odd and greater than 1, for example: 3, 5, 7 ... @note The median filter uses cv.BORDER\_REPLICATE internally to cope with border pixels. ## Try it \\htmlonly \\endhtmlonly ### 4\. Bilateral Filtering **cv.bilateralFilter()** is highly effective in noise removal while keeping edges sharp. But the operation is slower compared to other filters. We already saw that gaussian filter takes the a neighbourhood around the pixel and find its gaussian weighted average. This gaussian filter is a function of space alone, that is, nearby pixels are considered while filtering. It doesn't consider whether pixels have almost same intensity. It doesn't consider whether pixel is an edge pixel or not. So it blurs the edges also, which we don't want to do. Bilateral filter also takes a gaussian filter in space, but one more gaussian filter which is a function of pixel difference. Gaussian function of space make sure only nearby pixels are considered for blurring while gaussian function of intensity difference make sure only those pixels with similar intensity to central pixel is considered for blurring. So it preserves the edges since pixels at edges will have large intensity variation. We use the function: **cv.bilateralFilter (src, dst, d, sigmaColor, sigmaSpace, borderType = cv.BORDER\_DEFAULT)** @param src source 8-bit or floating-point, 1-channel or 3-channel image. @param dst output image of the same size and type as src. @param d diameter of each pixel neighborhood that is used during filtering. If it is non-positive, it is computed from sigmaSpace. @param sigmaColor filter sigma in the color space. A larger value of the parameter means that farther colors within the pixel neighborhood will be mixed together, resulting in larger areas of semi-equal color. @param sigmaSpace filter sigma in the coordinate space. A larger value of the parameter means that farther pixels will influence each other as long as their colors are close enough. When d>0, it specifies the neighborhood size regardless of sigmaSpace. Otherwise, d is proportional to sigmaSpace. @param borderType border mode used to extrapolate pixels outside of the image(see cv.BorderTypes). @note For simplicity, you can set the 2 sigma values to be the same. If they are small (< 10), the filter will not have much effect, whereas if they are large (> 150), they will have a very strong effect, making the image look "cartoonish". Large filters (d > 5) are very slow, so it is recommended to use d=5 for real-time applications, and perhaps d=9 for offline applications that need heavy noise filtering. ## Try it \\htmlonly \\endhtmlonly ## [Js Geometric Transformations](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_geometric_transformations/js_geometric_transformations/) Contents opencv5 Js Geometric Transformations OpenCV 5 Js Geometric Transformations # Geometric Transformations of Images {#tutorial\_js\_geometric\_transformations} ## Goals - Learn how to apply different geometric transformation to images like translation, rotation, affine transformation etc. - You will learn these functions: **cv.resize**, **cv.warpAffine**, **cv.getAffineTransform** and **cv.warpPerspective** ## Transformations ### Scaling Scaling is just resizing of the image. OpenCV comes with a function **cv.resize()** for this purpose. The size of the image can be specified manually, or you can specify the scaling factor. Different interpolation methods are used. Preferable interpolation methods are **cv.INTER\_AREA** for shrinking and **cv.INTER\_CUBIC** (slow) & **cv.INTER\_LINEAR** for zooming. We use the function: **cv.resize (src, dst, dsize, fx = 0, fy = 0, interpolation = cv.INTER\_LINEAR)** @param src input image @param dst output image; it has the size dsize (when it is non-zero) or the size computed from src.size(), fx, and fy; the type of dst is the same as of src. @param dsize output image size; if it equals zero, it is computed as: \\f\[𝚍𝚜𝚒𝚣𝚎 = 𝚂𝚒𝚣𝚎(𝚛𝚘𝚞𝚗𝚍(𝚏𝚡_𝚜𝚛𝚌.𝚌𝚘𝚕𝚜), 𝚛𝚘𝚞𝚗𝚍(𝚏𝚢_𝚜𝚛𝚌.𝚛𝚘𝚠𝚜))\\f\] Either dsize or both fx and fy must be non-zero. @param fx scale factor along the horizontal axis; when it equals 0, it is computed as \\f\[(𝚍𝚘𝚞𝚋𝚕𝚎)𝚍𝚜𝚒𝚣𝚎.𝚠𝚒𝚍𝚝𝚑/𝚜𝚛𝚌.𝚌𝚘𝚕𝚜\\f\] @param fy scale factor along the vertical axis; when it equals 0, it is computed as \\f\[(𝚍𝚘𝚞𝚋𝚕𝚎)𝚍𝚜𝚒𝚣𝚎.𝚑𝚎𝚒𝚐𝚑𝚝/𝚜𝚛𝚌.𝚛𝚘𝚠𝚜\\f\] @param interpolation interpolation method(see **cv.InterpolationFlags**) ## Try it \\htmlonly \\endhtmlonly ### Translation Translation is the shifting of object's location. If you know the shift in (x,y) direction, let it be \\f$(t\_x,t\_y)\\f$, you can create the transformation matrix \\f$\\textbf{M}\\f$ as follows: \\f\[M = \\begin{bmatrix} 1 & 0 & t\_x \\ 0 & 1 & t\_y \\end{bmatrix}\\f\] We use the function: **cv.warpAffine (src, dst, M, dsize, flags = cv.INTER\_LINEAR, borderMode = cv.BORDER\_CONSTANT, borderValue = new cv.Scalar())** @param src input image. @param dst output image that has the size dsize and the same type as src. @param Mat 2 × 3 transformation matrix(cv.CV\_64FC1 type). @param dsize size of the output image. @param flags combination of interpolation methods(see cv.InterpolationFlags) and the optional flag WARP\_INVERSE\_MAP that means that M is the inverse transformation ( 𝚍𝚜𝚝→𝚜𝚛𝚌 ) @param borderMode pixel extrapolation method (see cv.BorderTypes); when borderMode = BORDER\_TRANSPARENT, it means that the pixels in the destination image corresponding to the "outliers" in the source image are not modified by the function. @param borderValue value used in case of a constant border; by default, it is 0. rows. ## Try it \\htmlonly \\endhtmlonly ### Rotation Rotation of an image for an angle \\f$\\theta\\f$ is achieved by the transformation matrix of the form \\f\[M = \\begin{bmatrix} cos\\theta & -sin\\theta \\ sin\\theta & cos\\theta \\end{bmatrix}\\f\] But OpenCV provides scaled rotation with adjustable center of rotation so that you can rotate at any location you prefer. Modified transformation matrix is given by \\f\[\\begin{bmatrix} \\alpha & \\beta & (1- \\alpha ) \\cdot center.x - \\beta \\cdot center.y \\ - \\beta & \\alpha & \\beta \\cdot center.x + (1- \\alpha ) \\cdot center.y \\end{bmatrix}\\f\] where: \\f\[\\begin{array}{l} \\alpha = scale \\cdot \\cos \\theta , \\ \\beta = scale \\cdot \\sin \\theta \\end{array}\\f\] We use the function: **cv.getRotationMatrix2D (center, angle, scale)** @param center center of the rotation in the source image. @param angle rotation angle in degrees. Positive values mean counter-clockwise rotation (the coordinate origin is assumed to be the top-left corner). @param scale isotropic scale factor. ## Try it \\htmlonly \\endhtmlonly ### Affine Transformation In affine transformation, all parallel lines in the original image will still be parallel in the output image. To find the transformation matrix, we need three points from input image and their corresponding locations in output image. Then **cv.getAffineTransform** will create a 2x3 matrix which is to be passed to **cv.warpAffine**. We use the function: **cv.getAffineTransform (src, dst)** @param src three points(\[3, 1\] size and cv.CV\_32FC2 type) from input imag. @param dst three corresponding points(\[3, 1\] size and cv.CV\_32FC2 type) in output image. ## Try it \\htmlonly \\endhtmlonly ### Perspective Transformation For perspective transformation, you need a 3x3 transformation matrix. Straight lines will remain straight even after the transformation. To find this transformation matrix, you need 4 points on the input image and corresponding points on the output image. Among these 4 points, 3 of them should not be collinear. Then transformation matrix can be found by the function **cv.getPerspectiveTransform**. Then apply **cv.warpPerspective** with this 3x3 transformation matrix. We use the functions: **cv.warpPerspective (src, dst, M, dsize, flags = cv.INTER\_LINEAR, borderMode = cv.BORDER\_CONSTANT, borderValue = new cv.Scalar())** @param src input image. @param dst output image that has the size dsize and the same type as src. @param Mat 3 × 3 transformation matrix(cv.CV\_64FC1 type). @param dsize size of the output image. @param flags combination of interpolation methods (cv.INTER\_LINEAR or cv.INTER\_NEAREST) and the optional flag WARP\_INVERSE\_MAP, that sets M as the inverse transformation (𝚍𝚜𝚝→𝚜𝚛𝚌). @param borderMode pixel extrapolation method (cv.BORDER\_CONSTANT or cv.BORDER\_REPLICATE). @param borderValue value used in case of a constant border; by default, it is 0. **cv.getPerspectiveTransform (src, dst)** @param src coordinates of quadrangle vertices in the source image. @param dst coordinates of the corresponding quadrangle vertices in the destination image. ## Try it \\htmlonly \\endhtmlonly ## [Js Grabcut](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_grabcut/js_grabcut/) Contents opencv5 Js Grabcut OpenCV 5 Js Grabcut # Foreground Extraction using GrabCut Algorithm {#tutorial\_js\_grabcut} ## Goal - We will learn GrabCut algorithm to extract foreground in images ## Theory GrabCut algorithm was designed by Carsten Rother, Vladimir Kolmogorov & Andrew Blake from Microsoft Research Cambridge, UK. in their paper, ["GrabCut": interactive foreground extraction using iterated graph cuts](http://dl.acm.org/citation.cfm?id=1015720) . An algorithm was needed for foreground extraction with minimal user interaction, and the result was GrabCut. How it works from user point of view ? Initially user draws a rectangle around the foreground region (foreground region should be completely inside the rectangle). Then algorithm segments it iteratively to get the best result. Done. But in some cases, the segmentation won't be fine, like, it may have marked some foreground region as background and vice versa. In that case, user need to do fine touch-ups. Just give some strokes on the images where some faulty results are there. Strokes basically says _"Hey, this region should be foreground, you marked it background, correct it in next iteration"_ or its opposite for background. Then in the next iteration, you get better results. What happens in background ? - User inputs the rectangle. Everything outside this rectangle will be taken as sure background (That is the reason it is mentioned before that your rectangle should include all the objects). Everything inside rectangle is unknown. Similarly any user input specifying foreground and background are considered as hard-labelling which means they won't change in the process. - Computer does an initial labelling depending on the data we gave. It labels the foreground and background pixels (or it hard-labels) - Now a Gaussian Mixture Model(GMM) is used to model the foreground and background. - Depending on the data we gave, GMM learns and create new pixel distribution. That is, the unknown pixels are labelled either probable foreground or probable background depending on its relation with the other hard-labelled pixels in terms of color statistics (It is just like clustering). - A graph is built from this pixel distribution. Nodes in the graphs are pixels. Additional two nodes are added, **Source node** and **Sink node**. Every foreground pixel is connected to Source node and every background pixel is connected to Sink node. - The weights of edges connecting pixels to source node/end node are defined by the probability of a pixel being foreground/background. The weights between the pixels are defined by the edge information or pixel similarity. If there is a large difference in pixel color, the edge between them will get a low weight. - Then a mincut algorithm is used to segment the graph. It cuts the graph into two separating source node and sink node with minimum cost function. The cost function is the sum of all weights of the edges that are cut. After the cut, all the pixels connected to Source node become foreground and those connected to Sink node become background. - The process is continued until the classification converges. It is illustrated in below image (Image Courtesy: [http://www.cs.ru.ac.za/research/g02m1682/](http://www.cs.ru.ac.za/research/g02m1682/)) ## Demo We use the function: **cv.grabCut (image, mask, rect, bgdModel, fgdModel, iterCount, mode = cv.GC\_EVAL)** @param image input 8-bit 3-channel image. @param mask input/output 8-bit single-channel mask. The mask is initialized by the function when mode is set to GC\_INIT\_WITH\_RECT. Its elements may have one of the cv.grabCutClasses. @param rect ROI containing a segmented object. The pixels outside of the ROI are marked as "obvious background". The parameter is only used when mode==GC\_INIT\_WITH\_RECT. @param bgdModel temporary array for the background model. Do not modify it while you are processing the same image. @param fgdModel temporary arrays for the foreground model. Do not modify it while you are processing the same image. @param iterCount number of iterations the algorithm should make before returning the result. Note that the result can be refined with further calls with mode==GC\_INIT\_WITH\_MASK or mode==GC\_EVAL . @param mode operation mode that could be one of the cv::GrabCutModes ## Try it \\htmlonly \\endhtmlonly ## [Js Gradients](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_gradients/js_gradients/) Contents opencv5 Js Gradients OpenCV 5 Js Gradients # Image Gradients {#tutorial\_js\_gradients} ## Goal - Find Image gradients, edges etc - We will learn following functions : **cv.Sobel()**, **cv.Scharr()**, **cv.Laplacian()** etc ## Theory OpenCV provides three types of gradient filters or High-pass filters, Sobel, Scharr and Laplacian. We will see each one of them. ### 1\. Sobel and Scharr Derivatives Sobel operators is a joint Gaussian smoothing plus differentiation operation, so it is more resistant to noise. You can specify the direction of derivatives to be taken, vertical or horizontal (by the arguments, yorder and xorder respectively). You can also specify the size of kernel by the argument ksize. If ksize = -1, a 3x3 Scharr filter is used which gives better results than 3x3 Sobel filter. Please see the docs for kernels used. We use the functions: **cv.Sobel (src, dst, ddepth, dx, dy, ksize = 3, scale = 1, delta = 0, borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image of the same size and the same number of channels as src. @param ddepth output image depth(see cv.combinations); in the case of 8-bit input images it will result in truncated derivatives. @param dx order of the derivative x. @param dy order of the derivative y. @param ksize size of the extended Sobel kernel; it must be 1, 3, 5, or 7. @param scale optional scale factor for the computed derivative values. @param delta optional delta value that is added to the results prior to storing them in dst. @param borderType pixel extrapolation method(see cv.BorderTypes). **cv.Scharr (src, dst, ddepth, dx, dy, scale = 1, delta = 0, borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image of the same size and the same number of channels as src. @param ddepth output image depth(see cv.combinations). @param dx order of the derivative x. @param dy order of the derivative y. @param scale optional scale factor for the computed derivative values. @param delta optional delta value that is added to the results prior to storing them in dst. @param borderType pixel extrapolation method(see cv.BorderTypes). ## Try it \\htmlonly \\endhtmlonly ### 2\. Laplacian Derivatives It calculates the Laplacian of the image given by the relation, \\f$\\Delta src = \\frac{\\partial ^2{src}}{\\partial x^2} + \\frac{\\partial ^2{src}}{\\partial y^2}\\f$ where each derivative is found using Sobel derivatives. If ksize = 1, then following kernel is used for filtering: \\f\[kernel = \\begin{bmatrix} 0 & 1 & 0 \\ 1 & -4 & 1 \\ 0 & 1 & 0 \\end{bmatrix}\\f\] We use the function: **cv.Laplacian (src, dst, ddepth, ksize = 1, scale = 1, delta = 0, borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image of the same size and the same number of channels as src. @param ddepth output image depth. @param ksize aperture size used to compute the second-derivative filters. @param scale optional scale factor for the computed Laplacian values. @param delta optional delta value that is added to the results prior to storing them in dst. @param borderType pixel extrapolation method(see cv.BorderTypes). ## Try it \\htmlonly \\endhtmlonly ## One Important Matter! In our last example, output datatype is cv.CV\_8U. But there is a slight problem with that. Black-to-White transition is taken as Positive slope (it has a positive value) while White-to-Black transition is taken as a Negative slope (It has negative value). So when you convert data to cv.CV\_8U, all negative slopes are made zero. In simple words, you miss that edge. If you want to detect both edges, better option is to keep the output datatype to some higher forms, like cv.CV\_16S, cv.CV\_64F etc, take its absolute value and then convert back to cv.CV\_8U. Below code demonstrates this procedure for a horizontal Sobel filter and difference in results. ## Try it \\htmlonly \\endhtmlonly ## [Js Histogram Backprojection](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_histograms/js_histogram_backprojection/js_histogram_backprojection/) Contents opencv5 Js Histogram Backprojection OpenCV 5 Js Histogram Backprojection # Histogram - 3 : Histogram Backprojection {#tutorial\_js\_histogram\_backprojection} ## Goal - We will learn about histogram backprojection. ## Theory It was proposed by **Michael J. Swain , Dana H. Ballard** in their paper **Indexing via color histograms**. **What is it actually in simple words?** It is used for image segmentation or finding objects of interest in an image. In simple words, it creates an image of the same size (but single channel) as that of our input image, where each pixel corresponds to the probability of that pixel belonging to our object. In more simpler worlds, the output image will have our object of interest in more white compared to remaining part. Well, that is an intuitive explanation. (I can't make it more simpler). Histogram Backprojection is used with camshift algorithm etc. **How do we do it ?** We create a histogram of an image containing our object of interest (in our case, the ground, leaving player and other things). The object should fill the image as far as possible for better results. And a color histogram is preferred over grayscale histogram, because color of the object is a better way to define the object than its grayscale intensity. We then "back-project" this histogram over our test image where we need to find the object, ie in other words, we calculate the probability of every pixel belonging to the ground and show it. The resulting output on proper thresholding gives us the ground alone. ## Backprojection in OpenCV We use the functions: **cv.calcBackProject (images, channels, hist, dst, ranges, scale)** @param images source arrays. They all should have the same depth, cv.CV\_8U, cv.CV\_16U or cv.CV\_32F , and the same size. Each of them can have an arbitrary number of channels. @param channels the list of channels used to compute the back projection. The number of channels must match the histogram dimensionality. @param hist input histogram that can be dense or sparse. @param dst destination back projection array that is a single-channel array of the same size and depth as images\[0\]. @param ranges array of arrays of the histogram bin boundaries in each dimension(see cv.calcHist). @param scale optional scale factor for the output back projection. **cv.normalize (src, dst, alpha = 1, beta = 0, norm\_type = cv.NORM\_L2, dtype = -1, mask = new cv.Mat())** @param src input array. @param dst output array of the same size as src . @param alpha norm value to normalize to or the lower range boundary in case of the range normalization. @param beta upper range boundary in case of the range normalization; it is not used for the norm normalization. @param norm\_type normalization type (see cv.NormTypes). @param dtype when negative, the output array has the same type as src; otherwise, it has the same number of channels as src and the depth = CV\_MAT\_DEPTH(dtype). @param mask optional operation mask. ## Try it \\htmlonly \\endhtmlonly ## [Js Histogram Begins](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_histograms/js_histogram_begins/js_histogram_begins/) Contents opencv5 Js Histogram Begins OpenCV 5 Js Histogram Begins # Histograms - 1 : Find, Plot, Analyze !!! {#tutorial\_js\_histogram\_begins} ## Goal - Find histograms - Plot histograms - You will learn the function: **cv.calcHist()**. ## Theory So what is histogram ? You can consider histogram as a graph or plot, which gives you an overall idea about the intensity distribution of an image. It is a plot with pixel values (ranging from 0 to 255, not always) in X-axis and corresponding number of pixels in the image on Y-axis. It is just another way of understanding the image. By looking at the histogram of an image, you get intuition about contrast, brightness, intensity distribution etc of that image. Almost all image processing tools today, provides features on histogram. Below is an image from [Cambridge in Color website](http://www.cambridgeincolour.com/tutorials/histograms1.htm), and I recommend you to visit the site for more details. You can see the image and its histogram. (Remember, this histogram is drawn for grayscale image, not color image). Left region of histogram shows the amount of darker pixels in image and right region shows the amount of brighter pixels. From the histogram, you can see dark region is more than brighter region, and amount of midtones (pixel values in mid-range, say around 127) are very less. ## Find Histogram We use the function: **cv.calcHist (image, channels, mask, hist, histSize, ranges, accumulate = false)** @param image source arrays. They all should have the same depth, cv.CV\_8U, cv.CV\_16U or cv.CV\_32F , and the same size. Each of them can have an arbitrary number of channels. @param channels list of the dims channels used to compute the histogram. @param mask optional mask. If the matrix is not empty, it must be an 8-bit array of the same size as images\[i\] . The non-zero mask elements mark the array elements counted in the histogram. @param hist output histogram(cv.CV\_32F type), which is a dense or sparse dims -dimensional array. @param histSize array of histogram sizes in each dimension. @param ranges array of the dims arrays of the histogram bin boundaries in each dimension. @param accumulate accumulation flag. If it is set, the histogram is not cleared in the beginning when it is allocated. This feature enables you to compute a single histogram from several sets of arrays, or to update the histogram in time. ## Try it \\htmlonly \\endhtmlonly ## [Js Histogram Equalization](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_histograms/js_histogram_equalization/js_histogram_equalization/) Contents opencv5 Js Histogram Equalization OpenCV 5 Js Histogram Equalization # Histograms - 2: Histogram Equalization {#tutorial\_js\_histogram\_equalization} ## Goal - We will learn the concepts of histogram equalization and use it to improve the contrast of our images. ## Theory Consider an image whose pixel values are confined to some specific range of values only. For eg, brighter image will have all pixels confined to high values. But a good image will have pixels from all regions of the image. So you need to stretch this histogram to either ends (as given in below image, from wikipedia) and that is what Histogram Equalization does (in simple words). This normally improves the contrast of the image. I would recommend you to read the wikipedia page on [Histogram Equalization](http://en.wikipedia.org/wiki/Histogram_equalization) for more details about it. It has a very good explanation with worked out examples, so that you would understand almost everything after reading that. ## Histograms Equalization in OpenCV We use the function: **cv.equalizeHist (src, dst)** @param src source 8-bit single channel image. @param dst destination image of the same size and type as src. ## Try it \\htmlonly \\endhtmlonly ## CLAHE (Contrast Limited Adaptive Histogram Equalization) In **adaptive histogram equalization**, image is divided into small blocks called "tiles" (tileSize is 8x8 by default in OpenCV). Then each of these blocks are histogram equalized as usual. So in a small area, histogram would confine to a small region (unless there is noise). If noise is there, it will be amplified. To avoid this, **contrast limiting** is applied. If any histogram bin is above the specified contrast limit (by default 40 in OpenCV), those pixels are clipped and distributed uniformly to other bins before applying histogram equalization. After equalization, to remove artifacts in tile borders, bilinear interpolation is applied. We use the class: **cv.CLAHE (clipLimit = 40, tileGridSize = new cv.Size(8, 8))** @param clipLimit threshold for contrast limiting. @param tileGridSize size of grid for histogram equalization. Input image will be divided into equally sized rectangular tiles. tileGridSize defines the number of tiles in row and column. @note Don't forget to delete CLAHE! ## Try it \\htmlonly \\endhtmlonly ## [Js Table Of Contents Histograms](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_histograms/js_table_of_contents_histograms/) Contents opencv5 Js Table Of Contents Histograms OpenCV 5 Js Table Of Contents Histograms # Histograms in OpenCV.js {#tutorial\_js\_table\_of\_contents\_histograms} - @subpage tutorial\_js\_histogram\_begins Learn the basics of histograms - @subpage tutorial\_js\_histogram\_equalization Learn to Equalize Histograms to get better contrast for images - @subpage tutorial\_js\_histogram\_backprojection Learn histogram backprojection to segment colored objects ## [Js Houghcircles](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_houghcircles/js_houghcircles/) Contents opencv5 Js Houghcircles OpenCV 5 Js Houghcircles # Hough Circle Transform {#tutorial\_js\_houghcircles} ## Goal - We will learn to use Hough Transform to find circles in an image. - We will learn these functions: **cv.HoughCircles()** ## Theory A circle is represented mathematically as \\f$(x-x\_{center})^2 + (y - y\_{center})^2 = r^2\\f$ where \\f$(x\_{center},y\_{center})\\f$ is the center of the circle, and \\f$r\\f$ is the radius of the circle. From equation, we can see we have 3 parameters, so we need a 3D accumulator for hough transform, which would be highly ineffective. So OpenCV uses more trickier method, **Hough Gradient Method** which uses the gradient information of edges. We use the function: **cv.HoughCircles (image, circles, method, dp, minDist, param1 = 100, param2 = 100, minRadius = 0, maxRadius = 0)** @param image 8-bit, single-channel, grayscale input image. @param circles output vector of found circles(cv.CV\_32FC3 type). Each vector is encoded as a 3-element floating-point vector (x,y,radius) . @param method detection method(see cv.HoughModes). Currently, the only implemented method is HOUGH\_GRADIENT @param dp inverse ratio of the accumulator resolution to the image resolution. For example, if dp = 1 , the accumulator has the same resolution as the input image. If dp = 2 , the accumulator has half as big width and height. @param minDist minimum distance between the centers of the detected circles. If the parameter is too small, multiple neighbor circles may be falsely detected in addition to a true one. If it is too large, some circles may be missed. @param param1 first method-specific parameter. In case of HOUGH\_GRADIENT , it is the higher threshold of the two passed to the Canny edge detector (the lower one is twice smaller). @param param2 second method-specific parameter. In case of HOUGH\_GRADIENT , it is the accumulator threshold for the circle centers at the detection stage. The smaller it is, the more false circles may be detected. Circles, corresponding to the larger accumulator values, will be returned first. @param minRadius minimum circle radius. @param maxRadius maximum circle radius. ## Try it \\htmlonly \\endhtmlonly ## [Js Houghlines](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_houghlines/js_houghlines/) Contents opencv5 Js Houghlines OpenCV 5 Js Houghlines # Hough Line Transform {#tutorial\_js\_houghlines} ## Goal - We will understand the concept of the Hough Transform. - We will learn how to use it to detect lines in an image. - We will learn the following functions: **cv.HoughLines()**, **cv.HoughLinesP()** ## Theory The Hough Transform is a popular technique to detect any shape, if you can represent that shape in a mathematical form. It can detect the shape even if it is broken or distorted a little bit. We will see how it works for a line. A line can be represented as \\f$y = mx+c\\f$ or in a parametric form, as \\f$\\rho = x \\cos \\theta + y \\sin \\theta\\f$ where \\f$\\rho\\f$ is the perpendicular distance from the origin to the line, and \\f$\\theta\\f$ is the angle formed by this perpendicular line and the horizontal axis measured in counter-clockwise (That direction varies on how you represent the coordinate system. This representation is used in OpenCV). Check the image below: So if the line is passing below the origin, it will have a positive rho and an angle less than 180. If it is going above the origin, instead of taking an angle greater than 180, the angle is taken less than 180, and rho is taken negative. Any vertical line will have 0 degree and horizontal lines will have 90 degree. Now let's see how the Hough Transform works for lines. Any line can be represented in these two terms, \\f$(\\rho, \\theta)\\f$. So first it creates a 2D array or accumulator (to hold the values of the two parameters) and it is set to 0 initially. Let rows denote the \\f$\\rho\\f$ and columns denote the \\f$\\theta\\f$. Size of array depends on the accuracy you need. Suppose you want the accuracy of angles to be 1 degree, you will need 180 columns. For \\f$\\rho\\f$, the maximum distance possible is the diagonal length of the image. So taking one pixel accuracy, the number of rows can be the diagonal length of the image. Consider a 100x100 image with a horizontal line at the middle. Take the first point of the line. You know its (x,y) values. Now in the line equation, put the values \\f$\\theta = 0,1,2,....,180\\f$ and check the \\f$\\rho\\f$ you get. For every \\f$(\\rho, \\theta)\\f$ pair, you increment value by one in our accumulator in its corresponding \\f$(\\rho, \\theta)\\f$ cells. So now in accumulator, the cell (50,90) = 1 along with some other cells. Now take the second point on the line. Do the same as above. Increment the values in the cells corresponding to \\f$(\\rho, \\theta)\\f$ you got. This time, the cell (50,90) = 2. What you actually do is voting the \\f$(\\rho, \\theta)\\f$ values. You continue this process for every point on the line. At each point, the cell (50,90) will be incremented or voted up, while other cells may or may not be voted up. This way, at the end, the cell (50,90) will have maximum votes. So if you search the accumulator for maximum votes, you get the value (50,90) which says, there is a line in this image at a distance 50 from the origin and at angle 90 degrees. It is well shown in the below animation (Image Courtesy: [Amos Storkey](http://homepages.inf.ed.ac.uk/amos/hough.html) ) This is how hough transform works for lines. It is simple. Below is an image which shows the accumulator. Bright spots at some locations denote they are the parameters of possible lines in the image. (Image courtesy: [Wikipedia](http://en.wikipedia.org/wiki/Hough_transform) ) # Hough Transform in OpenCV Everything explained above is encapsulated in the OpenCV function, **cv.HoughLines()**. It simply returns an array of (\\f$(\\rho, \\theta)\\f$ values. \\f$\\rho\\f$ is measured in pixels and \\f$\\theta\\f$ is measured in radians. First parameter, Input image should be a binary image, so apply threshold or use canny edge detection before applying hough transform. We use the function: **cv.HoughLines (image, lines, rho, theta, threshold, srn = 0, stn = 0, min\_theta = 0, max\_theta = Math.PI)** @param image 8-bit, single-channel binary source image. The image may be modified by the function. @param lines output vector of lines(cv.32FC2 type). Each line is represented by a two-element vector (ρ,θ) . ρ is the distance from the coordinate origin (0,0). θ is the line rotation angle in radians. @param rho distance resolution of the accumulator in pixels. @param theta angle resolution of the accumulator in radians. @param threshold accumulator threshold parameter. Only those lines are returned that get enough votes @param srn for the multi-scale Hough transform, it is a divisor for the distance resolution rho . The coarse accumulator distance resolution is rho and the accurate accumulator resolution is rho/srn . If both srn=0 and stn=0 , the classical Hough transform is used. Otherwise, both these parameters should be positive. @param stn for the multi-scale Hough transform, it is a divisor for the distance resolution theta. @param min\_theta for standard and multi-scale Hough transform, minimum angle to check for lines. Must fall between 0 and max\_theta. @param max\_theta for standard and multi-scale Hough transform, maximum angle to check for lines. Must fall between min\_theta and CV\_PI. ## Try it \\htmlonly \\endhtmlonly ## Probabilistic Hough Transform In the hough transform, you can see that even for a line with two arguments, it takes a lot of computation. Probabilistic Hough Transform is an optimization of the Hough Transform we saw. It doesn't take all the points into consideration. Instead, it takes only a random subset of points which is sufficient for line detection. Just we have to decrease the threshold. See image below which compares Hough Transform and Probabilistic Hough Transform in Hough space. (Image Courtesy : [Franck Bettinger's home page](http://phdfb1.free.fr/robot/mscthesis/node14.html) ) OpenCV implementation is based on Robust Detection of Lines Using the Progressive Probabilistic Hough Transform by Matas, J. and Galambos, C. and Kittler, J.V. @cite Matas00. We use the function: **cv.HoughLinesP (image, lines, rho, theta, threshold, minLineLength = 0, maxLineGap = 0)** @param image 8-bit, single-channel binary source image. The image may be modified by the function. @param lines output vector of lines(cv.32SC4 type). Each line is represented by a 4-element vector (x1,y1,x2,y2) ,where (x1,y1) and (x2,y2) are the ending points of each detected line segment. @param rho distance resolution of the accumulator in pixels. @param theta angle resolution of the accumulator in radians. @param threshold accumulator threshold parameter. Only those lines are returned that get enough votes @param minLineLength minimum line length. Line segments shorter than that are rejected. @param maxLineGap maximum allowed gap between points on the same line to link them. ## Try it \\htmlonly \\endhtmlonly ## [Js Imgproc Camera](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_imgproc_camera/js_imgproc_camera/) Contents opencv5 Js Imgproc Camera OpenCV 5 Js Imgproc Camera # Image Processing for Video Capture {#tutorial\_js\_imgproc\_camera} ## Goal - learn image processing for video capture. \\htmlonly \\endhtmlonly ## [Js Intelligent Scissors](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_intelligent_scissors/js_intelligent_scissors/) Contents opencv5 Js Intelligent Scissors OpenCV 5 Js Intelligent Scissors # Intelligent Scissors Demo {#tutorial\_js\_intelligent\_scissors} ## Goal - Here you can check how to use IntelligentScissors tool for image segmentation task. - Available methods and parameters: @ref cv::segmentation::IntelligentScissorsMB @note The feature is integrated into [CVAT](https://github.com/openvinotoolkit/cvat) annotation tool and you can try it online on [https://cvat.org](https://cvat.org) \\htmlonly \\endhtmlonly ## [Js Morphological Ops](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_morphological_ops/js_morphological_ops/) Contents opencv5 Js Morphological Ops OpenCV 5 Js Morphological Ops # Morphological Transformations {#tutorial\_js\_morphological\_ops} ## Goal - We will learn different morphological operations like Erosion, Dilation, Opening, Closing etc. - We will learn different functions like : **cv.erode()**, **cv.dilate()**, **cv.morphologyEx()** etc. ## Theory Morphological transformations are some simple operations based on the image shape. It is normally performed on binary images. It needs two inputs, one is our original image, second one is called **structuring element** or **kernel** which decides the nature of operation. Two basic morphological operators are Erosion and Dilation. Then its variant forms like Opening, Closing, Gradient etc also comes into play. We will see them one-by-one with help of following image: ### 1\. Erosion The basic idea of erosion is just like soil erosion only, it erodes away the boundaries of foreground object (Always try to keep foreground in white). So what it does? The kernel slides through the image (as in 2D convolution). A pixel in the original image (either 1 or 0) will be considered 1 only if all the pixels under the kernel is 1, otherwise it is eroded (made to zero). So what happends is that, all the pixels near boundary will be discarded depending upon the size of kernel. So the thickness or size of the foreground object decreases or simply white region decreases in the image. It is useful for removing small white noises (as we have seen in colorspace chapter), detach two connected objects etc. We use the function: **cv.erode (src, dst, kernel, anchor = new cv.Point(-1, -1), iterations = 1, borderType = cv.BORDER\_CONSTANT, borderValue = cv.morphologyDefaultBorderValue())** @param src input image; the number of channels can be arbitrary, but the depth should be one of cv.CV\_8U, cv.CV\_16U, cv.CV\_16S, cv.CV\_32F or cv.CV\_64F. @param dst output image of the same size and type as src. @param kernel structuring element used for erosion. @param anchor position of the anchor within the element; default value new cv.Point(-1, -1) means that the anchor is at the element center. @param iterations number of times erosion is applied. @param borderType pixel extrapolation method(see cv.BorderTypes). @param borderValue border value in case of a constant border ## Try it \\htmlonly \\endhtmlonly ### 2\. Dilation It is just opposite of erosion. Here, a pixel element is '1' if at least one pixel under the kernel is '1'. So it increases the white region in the image or size of foreground object increases. Normally, in cases like noise removal, erosion is followed by dilation. Because, erosion removes white noises, but it also shrinks our object. So we dilate it. Since noise is gone, they won't come back, but our object area increases. It is also useful in joining broken parts of an object. We use the function: **cv.dilate (src, dst, kernel, anchor = new cv.Point(-1, -1), iterations = 1, borderType = cv.BORDER\_CONSTANT, borderValue = cv.morphologyDefaultBorderValue())** @param src input image; the number of channels can be arbitrary, but the depth should be one of cv.CV\_8U, cv.CV\_16U, cv.CV\_16S, cv.CV\_32F or cv.CV\_64F. @param dst output image of the same size and type as src. @param kernel structuring element used for dilation. @param anchor position of the anchor within the element; default value new cv.Point(-1, -1) means that the anchor is at the element center. @param iterations number of times dilation is applied. @param borderType pixel extrapolation method(see cv.BorderTypes). @param borderValue border value in case of a constant border ## Try it \\htmlonly \\endhtmlonly ### 3\. Opening Opening is just another name of **erosion followed by dilation**. It is useful in removing noise. We use the function: **cv.morphologyEx (src, dst, op, kernel, anchor = new cv.Point(-1, -1), iterations = 1, borderType = cv.BORDER\_CONSTANT, borderValue = cv.morphologyDefaultBorderValue())** @param src source image. The number of channels can be arbitrary. The depth should be one of cv.CV\_8U, cv.CV\_16U, cv.CV\_16S, cv.CV\_32F or cv.CV\_64F @param dst destination image of the same size and type as source image. @param op type of a morphological operation, (see cv.MorphTypes). @param kernel structuring element. It can be created using cv.getStructuringElement. @param anchor anchor position with the kernel. Negative values mean that the anchor is at the kernel center. @param iterations number of times dilation is applied. @param borderType pixel extrapolation method(see cv.BorderTypes). @param borderValue border value in case of a constant border. The default value has a special meaning. ## Try it \\htmlonly \\endhtmlonly ### 4\. Closing Closing is reverse of Opening, **Dilation followed by Erosion**. It is useful in closing small holes inside the foreground objects, or small black points on the object. ## Try it \\htmlonly \\endhtmlonly ### 5\. Morphological Gradient It is the difference between dilation and erosion of an image. The result will look like the outline of the object. ## Try it \\htmlonly \\endhtmlonly ### 6\. Top Hat It is the difference between input image and Opening of the image. ## Try it \\htmlonly \\endhtmlonly ### 7\. Black Hat It is the difference between the closing of the input image and input image. ## Try it \\htmlonly \\endhtmlonly ## Structuring Element We manually created a structuring elements in the previous examples with help of cv.Mat.ones. It is rectangular shape. But in some cases, you may need elliptical/circular shaped kernels or diamond-shaped kernels. So for this purpose, OpenCV has a function, **cv.getStructuringElement()**. You just pass the shape and size of the kernel, you get the desired kernel. We use the function: **cv.getStructuringElement (shape, ksize, anchor = new cv.Point(-1, -1))** @param shape element shape that could be one of cv.MorphShapes @param ksize size of the structuring element. @param anchor anchor position within the element. The default value \[−1,−1\] means that the anchor is at the center. Note that only the shape of a cross-shaped element depends on the anchor position. In other cases the anchor just regulates how much the result of the morphological operation is shifted. ## Try it \\htmlonly \\endhtmlonly ## [Js Pyramids](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_pyramids/js_pyramids/) Contents opencv5 Js Pyramids OpenCV 5 Js Pyramids # Image Pyramids {#tutorial\_js\_pyramids} ## Goal - We will learn about Image Pyramids - We will learn these functions: **cv.pyrUp()**, **cv.pyrDown()** ## Theory Normally, we used to work with an image of constant size. But on some occasions, we need to work with (the same) images in different resolution. For example, while searching for something in an image, like face, we are not sure at what size the object will be present in said image. In that case, we will need to create a set of the same image with different resolutions and search for object in all of them. These set of images with different resolutions are called **Image Pyramids** (because when they are kept in a stack with the highest resolution image at the bottom and the lowest resolution image at top, it looks like a pyramid). There are two kinds of Image Pyramids. 1) **Gaussian Pyramid** and 2) **Laplacian Pyramids** Higher level (Low resolution) in a Gaussian Pyramid is formed by removing consecutive rows and columns in Lower level (higher resolution) image. Then each pixel in higher level is formed by the contribution from 5 pixels in underlying level with gaussian weights. By doing so, a \\f$M \\times N\\f$ image becomes \\f$M/2 \\times N/2\\f$ image. So area reduces to one-fourth of original area. It is called an Octave. The same pattern continues as we go upper in pyramid (ie, resolution decreases). Similarly while expanding, area becomes 4 times in each level. We can find Gaussian pyramids using **cv.pyrDown()** and **cv.pyrUp()** functions. Laplacian Pyramids are formed from the Gaussian Pyramids. There is no exclusive function for that. Laplacian pyramid images are like edge images only. Most of its elements are zeros. They are used in image compression. A level in Laplacian Pyramid is formed by the difference between that level in Gaussian Pyramid and expanded version of its upper level in Gaussian Pyramid. ## Downsample We use the function: **cv.pyrDown (src, dst, dstsize = new cv.Size(0, 0), borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image; it has the specified size and the same type as src. @param dstsize size of the output image. @param borderType pixel extrapolation method(see cv.BorderTypes, cv.BORDER\_CONSTANT isn't supported). ## Try it \\htmlonly \\endhtmlonly ## Upsample We use the function: **cv.pyrUp (src, dst, dstsize = new cv.Size(0, 0), borderType = cv.BORDER\_DEFAULT)** @param src input image. @param dst output image; it has the specified size and the same type as src. @param dstsize size of the output image. @param borderType pixel extrapolation method(see cv.BorderTypes, only cv.BORDER\_DEFAULT is supported). ## Try it \\htmlonly \\endhtmlonly ## [Js Table Of Contents Imgproc](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_table_of_contents_imgproc/) Contents opencv5 Js Table Of Contents Imgproc OpenCV 5 Js Table Of Contents Imgproc # Image Processing {#tutorial\_js\_table\_of\_contents\_imgproc} - @subpage tutorial\_js\_colorspaces Learn how to change images between different color spaces. - @subpage tutorial\_js\_geometric\_transformations Learn how to apply different geometric transformations to images like rotation, translation etc. - @subpage tutorial\_js\_thresholding Learn how to convert images to binary images using global thresholding, Adaptive thresholding, Otsu's binarization etc. - @subpage tutorial\_js\_filtering Learn how to blur the images, filter the images with custom kernels etc. - @subpage tutorial\_js\_morphological\_ops Learn about morphological transformations like Erosion, Dilation, Opening, Closing etc. - @subpage tutorial\_js\_gradients Learn how to find image gradients, edges etc. - @subpage tutorial\_js\_canny Learn how to find edges with Canny Edge Detection. - @subpage tutorial\_js\_pyramids Learn about image pyramids and how to use them for image blending. - @subpage tutorial\_js\_table\_of\_contents\_contours Learn about Contours in OpenCV.js. - @subpage tutorial\_js\_table\_of\_contents\_histograms Learn about histograms in OpenCV.js. - @subpage tutorial\_js\_table\_of\_contents\_transforms Learn different Image Transforms in OpenCV.js like Fourier Transform, Cosine Transform etc. - @subpage tutorial\_js\_template\_matching Learn how to search for an object in an image using Template Matching. - @subpage tutorial\_js\_houghlines Learn how to detect lines in an image. - @subpage tutorial\_js\_houghcircles Learn how to detect circles in an image. - @subpage tutorial\_js\_watershed Learn how to segment images with watershed segmentation. - @subpage tutorial\_js\_grabcut Learn how to extract foreground with GrabCut algorithm. - @subpage tutorial\_js\_imgproc\_camera Learn image processing for video capture. - @subpage tutorial\_js\_intelligent\_scissors Learn how to use IntelligentScissors tool for image segmentation task. ## [Js Template Matching](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_template_matching/js_template_matching/) Contents opencv5 Js Template Matching OpenCV 5 Js Template Matching # Template Matching {#tutorial\_js\_template\_matching} ## Goals - To find objects in an image using Template Matching - You will learn these functions : **cv.matchTemplate()**, **cv.minMaxLoc()** ## Theory Template Matching is a method for searching and finding the location of a template image in a larger image. OpenCV comes with a function **cv.matchTemplate()** for this purpose. It simply slides the template image over the input image (as in 2D convolution) and compares the template and patch of input image under the template image. Several comparison methods are implemented in OpenCV. (You can check docs for more details). It returns a grayscale image, where each pixel denotes how much does the neighbourhood of that pixel match with template. If input image is of size (WxH) and template image is of size (wxh), output image will have a size of (W-w+1, H-h+1). Once you got the result, you can use **cv.minMaxLoc()** function to find where is the maximum/minimum value. Take it as the top-left corner of rectangle and take (w,h) as width and height of the rectangle. That rectangle is your region of template. @note If you are using cv.TM\_SQDIFF as comparison method, minimum value gives the best match. ## Template Matching in OpenCV We use the function: **cv.matchTemplate (image, templ, result, method, mask = new cv.Mat())** @param image image where the search is running. It must be 8-bit or 32-bit floating-point. @param templ searched template. It must be not greater than the source image and have the same data type. @param result map of comparison results. It must be single-channel 32-bit floating-point. @param method parameter specifying the comparison method(see cv.TemplateMatchModes). @param mask mask of searched template. It must have the same datatype and size with templ. It is not set by default. ## Try it \\htmlonly \\endhtmlonly ## [Js Thresholding](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_thresholding/js_thresholding/) Contents opencv5 Js Thresholding OpenCV 5 Js Thresholding # Image Thresholding {#tutorial\_js\_thresholding} ## Goal - In this tutorial, you will learn Simple thresholding, Adaptive thresholding, Otsu's thresholding etc. - You will learn these functions : **cv.threshold**, **cv.adaptiveThreshold** etc. ## Simple Thresholding Here, the matter is straight forward. If pixel value is greater than a threshold value, it is assigned one value (may be white), else it is assigned another value (may be black). We use the function: **cv.threshold (src, dst, thresh, maxval, type)** @param src input array. @param dst output array of the same size and type and the same number of channels as src. @param thresh threshold value. @param maxval maximum value to use with the cv.THRESH\_BINARY and cv.THRESH\_BINARY\_INV thresholding types. @param type thresholding type(see cv.ThresholdTypes). **thresholding type** - OpenCV provides different styles of thresholding and it is decided by the fourth parameter of the function. Different types are: - cv.THRESH\_BINARY - cv.THRESH\_BINARY\_INV - cv.THRESH\_TRUNC - cv.THRESH\_TOZERO - cv.THRESH\_OTSU - cv.THRESH\_TRIANGLE @note Input image should be single channel only in case of cv.THRESH\_OTSU or cv.THRESH\_TRIANGLE flags ## Try it \\htmlonly \\endhtmlonly ## Adaptive Thresholding In the previous section, we used a global value as threshold value. But it may not be good in all the conditions where image has different lighting conditions in different areas. In that case, we go for adaptive thresholding. In this, the algorithm calculate the threshold for a small regions of the image. So we get different thresholds for different regions of the same image and it gives us better results for images with varying illumination. We use the function: **cv.adaptiveThreshold (src, dst, maxValue, adaptiveMethod, thresholdType, blockSize, C)** @param src source 8-bit single-channel image. @param dst destination image of the same size and the same type as src. @param maxValue non-zero value assigned to the pixels for which the condition is satisfied @param adaptiveMethod adaptive thresholding algorithm to use. @param thresholdType thresholding type that must be either cv.THRESH\_BINARY or cv.THRESH\_BINARY\_INV. @param blockSize size of a pixel neighborhood that is used to calculate a threshold value for the pixel: 3, 5, 7, and so on. @param C constant subtracted from the mean or weighted mean (see the details below). Normally, it is positive but may be zero or negative as well. **adaptiveMethod** - It decides how thresholding value is calculated: - cv.ADAPTIVE\_THRESH\_MEAN\_C - cv.ADAPTIVE\_THRESH\_GAUSSIAN\_C ## Try it \\htmlonly \\endhtmlonly ## [Js Fourier Transform](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_transforms/js_fourier_transform/js_fourier_transform/) Contents opencv5 Js Fourier Transform OpenCV 5 Js Fourier Transform # Fourier Transform {#tutorial\_js\_fourier\_transform} ## Goal - To find the Fourier Transform of images using OpenCV - Some applications of Fourier Transform - We will learn following functions : **cv.dft()** etc ## Theory Fourier Transform is used to analyze the frequency characteristics of various filters. For images, **2D Discrete Fourier Transform (DFT)** is used to find the frequency domain. A fast algorithm called **Fast Fourier Transform (FFT)** is used for calculation of DFT. Details about these can be found in any image processing or signal processing textbooks. For a sinusoidal signal, \\f$x(t) = A \\sin(2 \\pi ft)\\f$, we can say \\f$f\\f$ is the frequency of signal, and if its frequency domain is taken, we can see a spike at \\f$f\\f$. If signal is sampled to form a discrete signal, we get the same frequency domain, but is periodic in the range \\f$\[- \\pi, \\pi\]\\f$ or \\f$\[0,2\\pi\]\\f$ (or \\f$\[0,N\]\\f$ for N-point DFT). You can consider an image as a signal which is sampled in two directions. So taking fourier transform in both X and Y directions gives you the frequency representation of image. More intuitively, for the sinusoidal signal, if the amplitude varies so fast in short time, you can say it is a high frequency signal. If it varies slowly, it is a low frequency signal. You can extend the same idea to images. Where does the amplitude varies drastically in images ? At the edge points, or noises. So we can say, edges and noises are high frequency contents in an image. If there is no much changes in amplitude, it is a low frequency component. Performance of DFT calculation is better for some array size. It is fastest when array size is power of two. The arrays whose size is a product of 2’s, 3’s, and 5’s are also processed quite efficiently. So if you are worried about the performance of your code, you can modify the size of the array to any optimal size (by padding zeros) before finding DFT. OpenCV provides a function, **cv.getOptimalDFTSize()** for this. Now we will see how to find the Fourier Transform. ## Fourier Transform in OpenCV Performance of DFT calculation is better for some array size. It is fastest when array size is power of two. The arrays whose size is a product of 2’s, 3’s, and 5’s are also processed quite efficiently. So if you are worried about the performance of your code, you can modify the size of the array to any optimal size (by padding zeros). So how do we find this optimal size ? OpenCV provides a function, cv.getOptimalDFTSize() for this. We use the functions: **cv.dft (src, dst, flags = 0, nonzeroRows = 0)** @param src input array that could be real or complex. @param dst output array whose size and type depends on the flags. @param flags transformation flags, representing a combination of the cv.DftFlags @param nonzeroRows when the parameter is not zero, the function assumes that only the first nonzeroRows rows of the input array (DFT\_INVERSE is not set) or only the first nonzeroRows of the output array (DFT\_INVERSE is set) contain non-zeros, thus, the function can handle the rest of the rows more efficiently and save some time; this technique is very useful for calculating array cross-correlation or convolution using DFT. **cv.getOptimalDFTSize (vecsize)** @param vecsize vector size. **cv.copyMakeBorder (src, dst, top, bottom, left, right, borderType, value = new cv.Scalar())** @param src input array that could be real or complex. @param dst output array whose size and type depends on the flags. @param top parameter specifying how many top pixels in each direction from the source image rectangle to extrapolate. @param bottom parameter specifying how many bottom pixels in each direction from the source image rectangle to extrapolate. @param left parameter specifying how many left pixels in each direction from the source image rectangle to extrapolate. @param right parameter specifying how many right pixels in each direction from the source image rectangle to extrapolate. @param borderType border type. @param value border value if borderType == cv.BORDER\_CONSTANT. **cv.magnitude (x, y, magnitude)** @param x floating-point array of x-coordinates of the vectors. @param y floating-point array of y-coordinates of the vectors; it must have the same size as x. @param magnitude output array of the same size and type as x. **cv.split (m, mv)** @param m input multi-channel array. @param mv output vector of arrays; the arrays themselves are reallocated, if needed. **cv.merge (mv, dst)** @param mv input vector of matrices to be merged; all the matrices in mv must have the same size and the same depth. @param dst output array of the same size and the same depth as mv\[0\]; The number of channels will be the total number of channels in the matrix array. ## Try it \\htmlonly \\endhtmlonly ## [Js Table Of Contents Transforms](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_transforms/js_table_of_contents_transforms/) Contents opencv5 Js Table Of Contents Transforms OpenCV 5 Js Table Of Contents Transforms # Image Transforms in OpenCV.js {#tutorial\_js\_table\_of\_contents\_transforms} - @subpage tutorial\_js\_fourier\_transform Learn to find the Fourier Transform of images ## [Js Watershed](https://docharvest.github.io/docs/opencv5/js_tutorials/js_imgproc/js_watershed/js_watershed/) Contents opencv5 Js Watershed OpenCV 5 Js Watershed # Image Segmentation with Watershed Algorithm {#tutorial\_js\_watershed} ## Goal - We will learn how to use marker-based image segmentation using watershed algorithm - We will learn: **cv.watershed()** ## Theory Any grayscale image can be viewed as a topographic surface where high intensity denotes peaks and hills while low intensity denotes valleys. You start filling every isolated valleys (local minima) with different colored water (labels). As the water rises, depending on the peaks (gradients) nearby, water from different valleys, obviously with different colors will start to merge. To avoid that, you build barriers in the locations where water merges. You continue the work of filling water and building barriers until all the peaks are under water. Then the barriers you created gives you the segmentation result. This is the "philosophy" behind the watershed. You can visit the [CMM webpage on watershed](https://people.cmm.minesparis.psl.eu/users/beucher/wtshed.html) to understand it with the help of some animations. But this approach gives you oversegmented result due to noise or any other irregularities in the image. So OpenCV implemented a marker-based watershed algorithm where you specify which are all valley points are to be merged and which are not. It is an interactive image segmentation. What we do is to give different labels for our object we know. Label the region which we are sure of being the foreground or object with one color (or intensity), label the region which we are sure of being background or non-object with another color and finally the region which we are not sure of anything, label it with 0. That is our marker. Then apply watershed algorithm. Then our marker will be updated with the labels we gave, and the boundaries of objects will have a value of -1. ## Code Below we will see an example on how to use the Distance Transform along with watershed to segment mutually touching objects. Consider the coins image below, the coins are touching each other. Even if you threshold it, it will be touching each other. We start with finding an approximate estimate of the coins. For that, we can use the Otsu's binarization. ## Try it \\htmlonly \\endhtmlonly Now we need to remove any small white noises in the image. For that we can use morphological opening. To remove any small holes in the object, we can use morphological closing. So, now we know for sure that region near to center of objects are foreground and region much away from the object are background. Only region we are not sure is the boundary region of coins. So we need to extract the area which we are sure they are coins. Erosion removes the boundary pixels. So whatever remaining, we can be sure it is coin. That would work if objects were not touching each other. But since they are touching each other, another good option would be to find the distance transform and apply a proper threshold. Next we need to find the area which we are sure they are not coins. For that, we dilate the result. Dilation increases object boundary to background. This way, we can make sure whatever region in background in result is really a background, since boundary region is removed. See the image below. ## Try it \\htmlonly \\endhtmlonly The remaining regions are those which we don't have any idea, whether it is coins or background. Watershed algorithm should find it. These areas are normally around the boundaries of coins where foreground and background meet (Or even two different coins meet). We call it border. It can be obtained from subtracting sure\_fg area from sure\_bg area. We use the function: **cv.distanceTransform (src, dst, distanceType, maskSize, labelType = cv.CV\_32F)** @param src 8-bit, single-channel (binary) source image. @param dst output image with calculated distances. It is a 8-bit or 32-bit floating-point, single-channel image of the same size as src. @param distanceType type of distance(see cv.DistanceTypes). @param maskSize size of the distance transform mask, see (cv.DistanceTransformMasks). @param labelType type of output image. It can be cv.CV\_8U or cv.CV\_32F. Type cv.CV\_8U can be used only for the first variant of the function and distanceType == DIST\_L1. ## Try it \\htmlonly \\endhtmlonly In the thresholded image, we get some regions of coins which we are sure of coins and they are detached now. (In some cases, you may be interested in only foreground segmentation, not in separating the mutually touching objects. In that case, you need not use distance transform, just erosion is sufficient. Erosion is just another method to extract sure foreground area, that's all.) ## Try it \\htmlonly \\endhtmlonly Now we know for sure which are region of coins, which are background and all. So we create marker (it is an array of same size as that of original image, but with int32 datatype) and label the regions inside it. The regions we know for sure (whether foreground or background) are labelled with any positive integers, but different integers, and the area we don't know for sure are just left as zero. For this we use **cv.connectedComponents()**. It labels background of the image with 0, then other objects are labelled with integers starting from 1. But we know that if background is marked with 0, watershed will consider it as unknown area. So we want to mark it with different integer. Instead, we will mark unknown region, defined by unknown, with 0. Now our marker is ready. It is time for final step, apply watershed. Then marker image will be modified. The boundary region will be marked with -1. We use the function: **cv.connectedComponents (image, labels, connectivity = 8, ltype = cv.CV\_32S)** @param image the 8-bit single-channel image to be labeled. @param labels destination labeled image(cv.CV\_32SC1 type). @param connectivity 8 or 4 for 8-way or 4-way connectivity respectively. @param ltype output image label type. Currently cv.CV\_32S and cv.CV\_16U are supported. We use the function: **cv.watershed (image, markers)** @param image input 8-bit 3-channel image. @param markers input/output 32-bit single-channel image (map) of markers. It should have the same size as image . ## Try it \\htmlonly \\endhtmlonly ## [Js Intro](https://docharvest.github.io/docs/opencv5/js_tutorials/js_setup/js_intro/js_intro/) Contents opencv5 Js Intro OpenCV 5 Js Intro # Introduction to OpenCV.js and Tutorials {#tutorial\_js\_intro} ## OpenCV OpenCV was created at Intel in 1999 by **Gary Bradski**. The first release came out in 2000. **Vadim Pisarevsky** joined Gary Bradski to manage Intel's Russian software OpenCV team. In 2005, OpenCV was used on Stanley; the vehicle that won the 2005 DARPA Grand Challenge. Later, its active development continued under the support of Willow Garage, with Gary Bradski and Vadim Pisarevsky leading the project. OpenCV now supports a multitude of algorithms related to Computer Vision and Machine Learning and is expanding day by day. OpenCV supports a wide variety of programming languages such as C++, Python, and Java, and is available on different platforms including Windows, Linux, OS X, Android, and iOS. Interfaces for high-speed GPU operations based on CUDA and OpenCL are also under active development. OpenCV.js brings OpenCV to the open web platform and makes it available to the JavaScript programmer. ## OpenCV.js: OpenCV for the JavaScript programmer Web is the most ubiquitous open computing platform. With HTML5 standards implemented in every browser, web applications are able to render online video with HTML5 video tags, capture webcam video via WebRTC API, and access each pixel of a video frame via canvas API. With abundance of available multimedia content, web developers are in need of a wide array of image and vision processing algorithms in JavaScript to build innovative applications. This requirement is even more essential for emerging applications on the web, such as Web Virtual Reality (WebVR) and Augmented Reality (WebAR). All of these use cases demand efficient implementations of computation-intensive vision kernels on web. [Emscripten](https://emscripten.org/) is an LLVM-to-JavaScript compiler. It takes LLVM bitcode - which can be generated from C/C++ using clang, and compiles that into asm.js or WebAssembly that can execute directly inside the web browsers. . Asm.js is a highly optimizable, low-level subset of JavaScript. Asm.js enables ahead-of-time compilation and optimization in JavaScript engine that provide near-to-native execution speed. WebAssembly is a new portable, size- and load-time-efficient binary format suitable for compilation to the web. WebAssembly aims to execute at native speed. WebAssembly is currently being designed as an open standard by W3C. OpenCV.js is a JavaScript binding for selected subset of OpenCV functions for the web platform. It allows emerging web applications with multimedia processing to benefit from the wide variety of vision functions available in OpenCV. OpenCV.js leverages Emscripten to compile OpenCV functions into asm.js or WebAssembly targets, and provides a JavaScript APIs for web application to access them. The future versions of the library will take advantage of acceleration APIs that are available on the Web such as SIMD and multi-threaded execution. OpenCV.js was initially created in Parallel Architectures and Systems Group at University of California Irvine (UCI) as a research project funded by Intel Corporation. OpenCV.js was further improved and integrated into the OpenCV project as part of Google Summer of Code 2017 program. ## OpenCV.js Tutorials OpenCV introduces a new set of tutorials that will guide you through various functions available in OpenCV.js. **This guide is mainly focused on OpenCV 3.x version**. The purpose of OpenCV.js tutorials is to: -# Help with adaptability of OpenCV in web development -# Help the web community, developers and computer vision researchers to interactively access a variety of web-based OpenCV examples to help them understand specific vision algorithms. Because OpenCV.js is able to run directly inside browser, the OpenCV.js tutorial web pages are intuitive and interactive. For example, using WebRTC API and evaluating JavaScript code would allow developers to change the parameters of CV functions and do live CV coding on web pages to see the results in real time. Prior knowledge of JavaScript and web application development is recommended to understand this guide. ## Contributors Below is the list of contributors of OpenCV.js bindings and tutorials. - Sajjad Taheri (Architect of the initial version and GSoC mentor, University of California, Irvine) - Congxiang Pan (GSoC student, Shanghai Jiao Tong University) - Gang Song (GSoC student, Shanghai Jiao Tong University) - Wenyao Gan (Student intern, Shanghai Jiao Tong University) - Mohammad Reza Haghighat (Project initiator & sponsor, Intel Corporation) - Ningxin Hu (Students' supervisor, Intel Corporation) ## [Js Nodejs](https://docharvest.github.io/docs/opencv5/js_tutorials/js_setup/js_nodejs/js_nodejs/) Contents opencv5 Js Nodejs OpenCV 5 Js Nodejs # Using OpenCV.js In Node.js {#tutorial\_js\_nodejs} ## Goals In this tutorial, you will learn: - Use OpenCV.js in a [Node.js](https://nodejs.org) application. - Load images with [jimp](https://www.npmjs.com/package/jimp) in order to use them with OpenCV.js. - Using [jsdom](https://www.npmjs.com/package/canvas) and [node-canvas](https://www.npmjs.com/package/canvas) to support `cv.imread()`, `cv.imshow()` - The basics of [emscripten](https://emscripten.org/) APIs, like [Module](https://emscripten.org/docs/api_reference/module.html) and [File System](https://emscripten.org/docs/api_reference/Filesystem-API.html) on which OpenCV.js is based. - Learn Node.js basics. Although this tutorial assumes the user knows JavaScript, experience with Node.js is not required. @note Besides giving instructions to run OpenCV.js in Node.js, another objective of this tutorial is to introduce users to the basics of [emscripten](https://emscripten.org/) APIs, like [Module](https://emscripten.org/docs/api_reference/module.html) and [File System](https://emscripten.org/docs/api_reference/Filesystem-API.html) and also Node.js. ## Minimal example Create a file `example1.js` with the following content: @code{.js} // Define a global variable 'Module' with a method 'onRuntimeInitialized': Module = { onRuntimeInitialized() { // this is our application: console.log(cv.getBuildInformation()) } } // Load 'opencv.js' assigning the value to the global variable 'cv' cv = require('./opencv.js') @endcode ### Execute it - Save the file as `example1.js`. - Make sure the file `opencv.js` is in the same folder. - Make sure [Node.js](https://nodejs.org) is installed on your system. The following command should print OpenCV build information: @code{.bash} node example1.js @endcode ### What just happened? - **In the first statement**:, by defining a global variable named 'Module', emscripten will call `Module.onRuntimeInitialized()` when the library is ready to use. Our program is in that method and uses the global variable `cv` just like in the browser. - The statement **"cv = require('./opencv.js')"** requires the file `opencv.js` and assign the return value to the global variable `cv`. `require()` which is a Node.js API, is used to load modules and files. In this case we load the file `opencv.js` form the current folder, and, as said previously emscripten will call `Module.onRuntimeInitialized()` when its ready. - See [emscripten Module API](https://emscripten.org/docs/api_reference/module.html) for more details. ## Working with images OpenCV.js doesn't support image formats so we can't load png or jpeg images directly. In the browser it uses the HTML DOM (like HTMLCanvasElement and HTMLImageElement to decode and decode images). In node.js we will need to use a library for this. In this example we use [jimp](https://www.npmjs.com/package/jimp), which supports common image formats and is pretty easy to use. ### Example setup Execute the following commands to create a new node.js package and install [jimp](https://www.npmjs.com/package/jimp) dependency: @code{.bash} mkdir project1 cd project1 npm init -y npm install jimp @endcode ### The example @code{.js} const Jimp = require('jimp'); async function onRuntimeInitialized(){ // load local image file with jimp. It supports jpg, png, bmp, tiff and gif: var jimpSrc = await Jimp.read('./lena.jpg'); // `jimpImage.bitmap` property has the decoded ImageData that we can use to create a cv:Mat var src = cv.matFromImageData(jimpSrc.bitmap); // following lines is copy&paste of opencv.js dilate tutorial: let dst = new cv.Mat(); let M = cv.Mat.ones(5, 5, cv.CV\_8U); let anchor = new cv.Point(-1, -1); cv.dilate(src, dst, M, anchor, 1, cv.BORDER\_CONSTANT, cv.morphologyDefaultBorderValue()); // Now that we are finish, we want to write `dst` to file `output.png`. For this we create a `Jimp` // image which accepts the image data as a [`Buffer`](https://nodejs.org/docs/latest-v10.x/api/buffer.html). // `write('output.png')` will write it to disk and Jimp infers the output format from given file name: new Jimp({ width: dst.cols, height: dst.rows, data: Buffer.from(dst.data) }) .write('output.png'); src.delete(); dst.delete(); } // Finally, load the open.js as before. The function `onRuntimeInitialized` contains our program. Module = { onRuntimeInitialized }; cv = require('./opencv.js'); @endcode ### Execute it - Save the file as `exampleNodeJimp.js`. - Make sure a sample image `lena.jpg` exists in the current directory. The following command should generate the file `output.png`: @code{.bash} node exampleNodeJimp.js @endcode ## Emulating HTML DOM and canvas As you might already seen, the rest of the examples use functions like `cv.imread()`, `cv.imshow()` to read and write images. Unfortunately as mentioned they won't work on Node.js since there is no HTML DOM. In this section, you will learn how to use [jsdom](https://www.npmjs.com/package/canvas) and [node-canvas](https://www.npmjs.com/package/canvas) to emulate the HTML DOM on Node.js so those functions work. ### Example setup As before, we create a Node.js project and install the dependencies we need: @code{.bash} mkdir project2 cd project2 npm init -y npm install canvas jsdom @endcode ### The example @code{.js} const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas'); const { JSDOM } = require('jsdom'); const { writeFileSync, existsSync, mkdirSync } = require("fs"); // This is our program. This time we use JavaScript async / await and promises to handle asynchronicity. (async () => { // before loading opencv.js we emulate a minimal HTML DOM. See the function declaration below. installDOM(); await loadOpenCV(); // using node-canvas, we an image file to an object compatible with HTML DOM Image and therefore with cv.imread() const image = await loadImage('./lena.jpg'); const src = cv.imread(image); const dst = new cv.Mat(); const M = cv.Mat.ones(5, 5, cv.CV\_8U); const anchor = new cv.Point(-1, -1); cv.dilate(src, dst, M, anchor, 1, cv.BORDER\_CONSTANT, cv.morphologyDefaultBorderValue()); // we create an object compatible HTMLCanvasElement const canvas = createCanvas(300, 300); cv.imshow(canvas, dst); writeFileSync('output.jpg', canvas.toBuffer('image/jpeg')); src.delete(); dst.delete(); })(); // Load opencv.js just like before but using Promise instead of callbacks: function loadOpenCV() { return new Promise(resolve => { global.Module = { onRuntimeInitialized: resolve }; global.cv = require('./opencv.js'); }); } // Using jsdom and node-canvas we define some global variables to emulate HTML DOM. // Although a complete emulation can be archived, here we only define those globals used // by cv.imread() and cv.imshow(). function installDOM() { const dom = new JSDOM(); global.document = dom.window.document; // The rest enables DOM image and canvas and is provided by node-canvas global.Image = Image; global.HTMLCanvasElement = Canvas; global.ImageData = ImageData; global.HTMLImageElement = Image; } @endcode ### Execute it - Save the file as `exampleNodeCanvas.js`. - Make sure a sample image `lena.jpg` exists in the current directory. The following command should generate the file `output.jpg`: @code{.bash} node exampleNodeCanvas.js @endcode ## Dealing with files In this tutorial you will learn how to configure emscripten so it uses the local filesystem for file operations instead of using memory. Also it tries to describe how [files are supported by emscripten applications](https://emscripten.org/docs/api_reference/Filesystem-API.html) Accessing the emscripten filesystem is often needed in OpenCV applications for example to load machine learning models such as the ones used in @ref tutorial\_dnn\_googlenet and @ref tutorial\_dnn\_javascript. ### Example setup Before the example, is worth consider first how files are handled in emscripten applications such as OpenCV.js. Remember that OpenCV library is written in C++ and the file opencv.js is just that C++ code being translated to JavaScript or WebAssembly by emscripten C++ compiler. These C++ sources use standard APIs to access the filesystem and the implementation often ends up in system calls that read a file in the hard drive. Since JavaScript applications in the browser don't have access to the local filesystem, [emscripten emulates a standard filesystem](https://emscripten.org/docs/api_reference/Filesystem-API.html) so compiled C++ code works out of the box. In the browser, this filesystem is emulated in memory while in Node.js there's also the possibility of using the local filesystem directly. This is often preferable since there's no need of copy file's content in memory. This section explains how to do just that, this is, configuring emscripten so files are accessed directly from our local filesystem and relative paths match files relative to the current local directory as expected. ### The example @code{.js} const { Canvas, createCanvas, Image, ImageData, loadImage } = require('canvas'); const { JSDOM } = require('jsdom'); const { writeFileSync, existsSync, mkdirSync } = require('fs'); const https = require('https'); (async () => { const createFileFromUrl = function (path, url, maxRedirects = 10) { console.log('Downloading ' + url + '...'); return new Promise((resolve, reject) => { const download = (url, redirectCount) => { if (redirectCount > maxRedirects) { reject(new Error('Too many redirects')); } else { let connection = https.get(url, (response) => { if (response.statusCode === 200) { let data = \[\]; response.on('data', (chunk) => { data.push(chunk); }); ``` response.on('end', () => { try { writeFileSync(path, Buffer.concat(data)); resolve(); } catch (err) { reject(new Error('Failed to write file ' + path)); } }); } else if (response.statusCode === 302 || response.statusCode === 301) { connection.abort(); download(response.headers.location, redirectCount + 1); } else { reject(new Error('Failed to load ' + url + ' status: ' + response.statusCode)); } }).on('error', (err) => { reject(new Error('Network Error: ' + err.message)); }); } }; download(url, 0); ``` }); }; if (!existsSync('./face\_detection\_yunet\_2023mar.onnx')) { await createFileFromUrl('./face\_detection\_yunet\_2023mar.onnx', '[https://media.githubusercontent.com/media/opencv/opencv\_zoo/main/models/face\_detection\_yunet/face\_detection\_yunet\_2023mar.onnx](https://media.githubusercontent.com/media/opencv/opencv_zoo/main/models/face_detection_yunet/face_detection_yunet_2023mar.onnx)') } if (!existsSync('./opencv.js')) { await createFileFromUrl('./opencv.js', '[https://docs.opencv.org/5.x/opencv.js](https://docs.opencv.org/5.x/opencv.js)') } if (!existsSync('./lena.jpg')) { await createFileFromUrl('./lena.jpg', '[https://docs.opencv.org/5.x/lena.jpg](https://docs.opencv.org/5.x/lena.jpg)') } await loadOpenCV(); const image = await loadImage('./lena.jpg'); const src = cv.imread(image); let srcBGR = new cv.Mat(); cv.cvtColor(src, srcBGR, cv.COLOR\_RGBA2BGR); // Load the deep learning model file. Notice how we reference local files using relative paths just // like we normally would do let netDet = new cv.FaceDetectorYN("./face\_detection\_yunet\_2023mar.onnx", "", new cv.Size(320, 320), 0.9, 0.3, 5000); netDet.setInputSize(new cv.Size(src.cols, src.rows)); let out = new cv.Mat(); netDet.detect(srcBGR, out); let faces = \[\]; for (let i = 0, n = out.data32F.length; i < n; i += 15) { let left = out.data32F\[i\]; let top = out.data32F\[i + 1\]; let right = (out.data32F\[i\] + out.data32F\[i + 2\]); let bottom = (out.data32F\[i + 1\] + out.data32F\[i + 3\]); left = Math.min(Math.max(0, left), src.cols - 1); top = Math.min(Math.max(0, top), src.rows - 1); right = Math.min(Math.max(0, right), src.cols - 1); bottom = Math.min(Math.max(0, bottom), src.rows - 1); if (left < right && top < bottom) { faces.push({ x: left, y: top, width: right - left, height: bottom - top, x1: out.data32F\[i + 4\] < 0 || out.data32F\[i + 4\] > src.cols - 1 ? -1 : out.data32F\[i + 4\], y1: out.data32F\[i + 5\] < 0 || out.data32F\[i + 5\] > src.rows - 1 ? -1 : out.data32F\[i + 5\], x2: out.data32F\[i + 6\] < 0 || out.data32F\[i + 6\] > src.cols - 1 ? -1 : out.data32F\[i + 6\], y2: out.data32F\[i + 7\] < 0 || out.data32F\[i + 7\] > src.rows - 1 ? -1 : out.data32F\[i + 7\], x3: out.data32F\[i + 8\] < 0 || out.data32F\[i + 8\] > src.cols - 1 ? -1 : out.data32F\[i + 8\], y3: out.data32F\[i + 9\] < 0 || out.data32F\[i + 9\] > src.rows - 1 ? -1 : out.data32F\[i + 9\], x4: out.data32F\[i + 10\] < 0 || out.data32F\[i + 10\] > src.cols - 1 ? -1 : out.data32F\[i + 10\], y4: out.data32F\[i + 11\] < 0 || out.data32F\[i + 11\] > src.rows - 1 ? -1 : out.data32F\[i + 11\], x5: out.data32F\[i + 12\] < 0 || out.data32F\[i + 12\] > src.cols - 1 ? -1 : out.data32F\[i + 12\], y5: out.data32F\[i + 13\] < 0 || out.data32F\[i + 13\] > src.rows - 1 ? -1 : out.data32F\[i + 13\], confidence: out.data32F\[i + 14\] }) } } out.delete(); faces.forEach(function(rect) { cv.rectangle(src, {x: rect.x, y: rect.y}, {x: rect.x + rect.width, y: rect.y + rect.height}, \[0, 255, 0, 255\]); if(rect.x1>0 && rect.y1>0) cv.circle(src, {x: rect.x1, y: rect.y1}, 2, \[255, 0, 0, 255\], 2) if(rect.x2>0 && rect.y2>0) cv.circle(src, {x: rect.x2, y: rect.y2}, 2, \[0, 0, 255, 255\], 2) if(rect.x3>0 && rect.y3>0) cv.circle(src, {x: rect.x3, y: rect.y3}, 2, \[0, 255, 0, 255\], 2) if(rect.x4>0 && rect.y4>0) cv.circle(src, {x: rect.x4, y: rect.y4}, 2, \[255, 0, 255, 255\], 2) if(rect.x5>0 && rect.y5>0) cv.circle(src, {x: rect.x5, y: rect.y5}, 2, \[0, 255, 255, 255\], 2) }); const canvas = createCanvas(image.width, image.height); cv.imshow(canvas, src); writeFileSync('output3.jpg', canvas.toBuffer('image/jpeg')); console.log('The result is saved.') src.delete(); srcBGR.delete(); })(); /\*\* - Loads opencv.js. - Installs HTML Canvas emulation to support `cv.imread()` and `cv.imshow` - Mounts given local folder `localRootDir` in emscripten filesystem folder `rootDir`. By default it will mount the local current directory in emscripten `/work` directory. This means that `/work/foo.txt` will be resolved to the local file `./foo.txt` - @param {string} rootDir The directory in emscripten filesystem in which the local filesystem will be mount. - @param {string} localRootDir The local directory to mount in emscripten filesystem. - @returns {Promise} resolved when the library is ready to use. \*/ function loadOpenCV(rootDir = '/work', localRootDir = process.cwd()) { if(global.Module && global.Module.onRuntimeInitialized && global.cv && global.cv.imread) { Promise.resolve() } return new Promise(resolve => { installDOM() global.Module = { onRuntimeInitialized() { // We change emscripten current work directory to 'rootDir' so relative paths are resolved // relative to the current local folder, as expected cv.FS.chdir(rootDir) resolve() }, preRun() { // preRun() is another callback like onRuntimeInitialized() but is called just before the // library code runs. Here we mount a local folder in emscripten filesystem and we want to // do this before the library is executed so the filesystem is accessible from the start const FS = global.Module.FS // create rootDir if it doesn't exists if(!FS.analyzePath(rootDir).exists) { FS.mkdir(rootDir); } // create localRootFolder if it doesn't exists if(!existsSync(localRootDir)) { mkdirSync(localRootDir, { recursive: true}); } // FS.mount() is similar to Linux/POSIX mount operation. It basically mounts an external // filesystem with given format, in given current filesystem directory. FS.mount(FS.filesystems.NODEFS, { root: localRootDir}, rootDir); } }; global.cv = require('./opencv.js') }); } function installDOM(){ const dom = new JSDOM(); global.document = dom.window.document; global.Image = Image; global.HTMLCanvasElement = Canvas; global.ImageData = ImageData; global.HTMLImageElement = Image; } @endcode ### Execute it - Save the file as `exampleNodeCanvasData.js`. - The files `face_detection_yunet_2023mar.onnx`, `lena.jpg` and `opencv.js` will be downloaded if they not present in project's directory. The following command should generate the file `output3.jpg` look the image below: @code{.bash} node exampleNodeCanvasData.js @endcode ## [Js Setup](https://docharvest.github.io/docs/opencv5/js_tutorials/js_setup/js_setup/js_setup/) Contents opencv5 Js Setup OpenCV 5 Js Setup # Build OpenCV.js {#tutorial\_js\_setup} @note You don't have to build your own copy if you simply want to start using it. Refer the Using Opencv.js tutorial for steps on getting a prebuilt copy from our releases or online documentation. ## Installing Emscripten [Emscripten](https://github.com/emscripten-core/emscripten) is an LLVM-to-JavaScript compiler. We will use Emscripten to build OpenCV.js. @note While this describes installation of required tools from scratch, there's a section below also describing an alternative procedure to perform the same build using docker containers which is often easier. To Install Emscripten, follow instructions of [Emscripten SDK](https://emscripten.org/docs/getting_started/downloads.html). For example: @code{.bash} ./emsdk update ./emsdk install latest ./emsdk activate latest @endcode After install, ensure the `EMSDK` environment is setup correctly. For example: @code{.bash} source ./emsdk\_env.sh echo ${EMSDK} @endcode Modern versions of Emscripten requires to use `emcmake` / `emmake` launchers: @code{.bash} emcmake sh -c 'echo ${EMSCRIPTEN}' @endcode The version 2.0.10 of emscripten is verified for latest WebAssembly. Please check the version of Emscripten to use the newest features of WebAssembly. For example: @code{.bash} ./emsdk update ./emsdk install 2.0.10 ./emsdk activate 2.0.10 @endcode ## Obtaining OpenCV Source Code You can use the latest stable OpenCV version or you can grab the latest snapshot from our [Git repository](https://github.com/opencv/opencv.git). ### Obtaining the Latest Stable OpenCV Version - Go to our [releases page](https://opencv.org/releases). - Download the source archive and unpack it. ### Obtaining the Cutting-edge OpenCV from the Git Repository Launch Git client and clone [OpenCV repository](http://github.com/opencv/opencv). For example: @code{.bash} git clone [https://github.com/opencv/opencv.git](https://github.com/opencv/opencv.git) @endcode @note It requires `git` installed in your development environment. ## Building OpenCV.js from Source \-# To build `opencv.js`, execute python script `/platforms/js/build_js.py `. The build script builds WebAssembly version by default(`--build_wasm` switch is kept by back-compatibility reason). By default everything is bundled into one JavaScript file by `base64` encoding the WebAssembly code. For production builds you can add `--disable_single_file` which will reduce total size by writing the WebAssembly code to a dedicated `.wasm` file which the generated JavaScript file will automatically load. ``` For example, to build in `build_js` directory: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js @endcode @note - It requires `python` and `cmake` installed in your development environment. - To build with Emscripten 4.0.20 or later, append --cmake_option="-DCMAKE_CXX_STANDARD=17" . Embind requires C++17 or later since Emscripten 4.0.20. @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --cmake_option="-DCMAKE_CXX_STANDARD=17" @endcode ``` \-# \[Optional\] To build the OpenCV.js loader, append `--build_loader`. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_loader @endcode @note The loader is implemented as a js file in the path `/bin/loader.js`. The loader utilizes the [WebAssembly Feature Detection](https://github.com/GoogleChromeLabs/wasm-feature-detect) to detect the features of the browser and load corresponding OpenCV.js automatically. To use it, you need to use the UMD version of [WebAssembly Feature Detection](https://github.com/GoogleChromeLabs/wasm-feature-detect) and introduce the `loader.js` in your Web application. Example Code: @code{.javascript} // Set paths configuration let pathsConfig = { wasm: "../../build_wasm/opencv.js", threads: "../../build_mt/opencv.js", simd: "../../build_simd/opencv.js", threadsSimd: "../../build_mtSIMD/opencv.js", } // Load OpenCV.js and use the pathsConfiguration and main function as the params. loadOpenCV(pathsConfig, main); @endcode ``` \-# \[optional\] To build documents, append `--build_doc` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_doc @endcode @note It requires `doxygen` installed in your development environment. ``` \-# \[optional\] To build tests, append `--build_test` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_test @endcode ``` \-# \[optional\] To enable OpenCV contrib modules append `--cmake_option="-DOPENCV_EXTRA_MODULES_PATH=/path/to/opencv_contrib/modules/"` ``` For example: @code{.bash} emcmake python ./platforms/js/build_js.py build_js --cmake_option="-DOPENCV_EXTRA_MODULES_PATH=opencv_contrib/modules" @endcode ``` \-# \[optional\] To enable WebNN backend, append `--webnn` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --webnn @endcode ``` ## Running OpenCV.js Tests Remember to launch the build command passing `--build_test` as mentioned previously. This will generate test source code ready to run together with `opencv.js` file in `build_js/bin` ### Manually in your browser To run tests, launch a local web server in `\/bin` folder. For example, node http-server which serves on `localhost:8080`. Navigate the web browser to `http://localhost:8080/tests.html`, which runs the unit tests automatically. Command example: @code{.sh} npx http-server build\_js/bin firefox [http://localhost:8080/tests.html](http://localhost:8080/tests.html) @endcode @note This snippet and the following require [Node.js](https://nodejs.org) to be installed. ### Headless with Puppeteer Alternatively tests can run with [GoogleChrome/puppeteer](https://github.com/GoogleChrome/puppeteer#readme) which is a version of Google Chrome that runs in the terminal (useful for Continuous integration like travis CI, etc) @code{.sh} cd build\_js/bin npm install npm install --no-save puppeteer # automatically downloads Chromium package node run\_puppeteer.js @endcode @note Checkout `node run_puppeteer --help` for more options to debug and reporting. @note The command `npm install` only needs to be executed once, since installs the tools dependencies; after that they are ready to use. @note Use `PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 npm install --no-save puppeteer` to skip automatic downloading of Chromium. You may specify own Chromium/Chrome binary through `PUPPETEER_EXECUTABLE_PATH=$(which google-chrome)` environment variable. **BEWARE**: Puppeteer is only guaranteed to work with the bundled Chromium, use at your own risk. ### Using Node.js. For example: @code{.sh} cd build\_js/bin npm install node tests.js @endcode @note If all tests are failed, then consider using Node.js from 8.x version (`lts/carbon` from `nvm`). \-# \[optional\] To build `opencv.js` with threads optimization, append `--threads` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --threads @endcode The default threads number is the logic core number of your device. You can use `cv.parallel_pthreads_set_threads_num(number)` to set threads number by yourself and use `cv.parallel_pthreads_get_threads_num()` to get the current threads number. @note You should build wasm version of `opencv.js` if you want to enable this optimization. And the threads optimization only works in browser, not in node.js. You need to enable the `WebAssembly threads support` feature first with your browser. For example, if you use Chrome, please enable this flag in chrome://flags. ``` \-# \[optional\] To build `opencv.js` with wasm simd optimization, append `--simd` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --simd @endcode The simd optimization is experimental as wasm simd is still in development. @note Now only emscripten LLVM upstream backend supports wasm simd, referring to https://emscripten.org/docs/porting/simd.html. So you need to setup upstream backend environment with the following command first: @code{.bash} ./emsdk update ./emsdk install latest-upstream ./emsdk activate latest-upstream source ./emsdk_env.sh @endcode @note You should build wasm version of `opencv.js` if you want to enable this optimization. For browser, you need to enable the `WebAssembly SIMD support` feature first. For example, if you use Chrome, please enable this flag in chrome://flags. For Node.js, you need to run script with flag `--experimental-wasm-simd`. @note The simd version of `opencv.js` built by latest LLVM upstream may not work with the stable browser or old version of Node.js. Please use the latest version of unstable browser or Node.js to get new features, like `Chrome Dev`. ``` \-# \[optional\] To build wasm intrinsics tests, append `--build_wasm_intrin_test` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_wasm --simd --build_wasm_intrin_test @endcode For wasm intrinsics tests, you can use the following function to test all the cases: @code{.js} cv.test_hal_intrin_all() @endcode And the failed cases will be logged in the JavaScript debug console. If you only want to test single data type of wasm intrinsics, you can use the following functions: @code{.js} cv.test_hal_intrin_uint8() cv.test_hal_intrin_int8() cv.test_hal_intrin_uint16() cv.test_hal_intrin_int16() cv.test_hal_intrin_uint32() cv.test_hal_intrin_int32() cv.test_hal_intrin_uint64() cv.test_hal_intrin_int64() cv.test_hal_intrin_float32() cv.test_hal_intrin_float64() @endcode ``` \-# \[optional\] To build performance tests, append `--build_perf` option. ``` For example: @code{.bash} emcmake python ./opencv/platforms/js/build_js.py build_js --build_perf @endcode To run performance tests, launch a local web server in \/bin folder. For example, node http-server which serves on `localhost:8080`. There are some kernels now in the performance test like `cvtColor`, `resize` and `threshold`. For example, if you want to test `threshold`, please navigate the web browser to `http://localhost:8080/perf/perf_imgproc/perf_threshold.html`. You need to input the test parameter like `(1920x1080, CV_8UC1, THRESH_BINARY)`, and then click the `Run` button to run the case. And if you don't input the parameter, it will run all the cases of this kernel. You can also run tests using Node.js. For example, run `threshold` with parameter `(1920x1080, CV_8UC1, THRESH_BINARY)`: @code{.sh} cd bin/perf npm install node perf_threshold.js --test_param_filter="(1920x1080, CV_8UC1, THRESH_BINARY)" @endcode ``` ## Building OpenCV.js with Docker Alternatively, the same build can be can be accomplished using [docker](https://www.docker.com/) containers which is often easier and more reliable, particularly in non linux systems. You only need to install [docker](https://www.docker.com/) on your system and use a popular container that provides a clean well tested environment for emscripten builds like this, that already has latest versions of all the necessary tools installed. So, make sure [docker](https://www.docker.com/) is installed in your system and running. The following shell script should work in Linux and MacOS: @code{.bash} git clone [https://github.com/opencv/opencv.git](https://github.com/opencv/opencv.git) cd opencv docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) emscripten/emsdk emcmake python3 ./platforms/js/build\_js.py build\_js @endcode In Windows use the following PowerShell command: @code{.bash} docker run --rm --workdir /src -v "$(get-location):/src" "emscripten/emsdk" emcmake python3 ./platforms/js/build\_js.py build\_js @endcode @warning The example uses latest version of emscripten. If the build fails you should try a version that is known to work fine which is `2.0.10` using the following command: @code{.bash} docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) emscripten/emsdk:2.0.10 emcmake python3 ./platforms/js/build\_js.py build\_js @endcode In Windows use the following PowerShell command: @code{.bash} docker run --rm --workdir /src -v "$(get-location):/src" "emscripten/emsdk:2.0.10" emcmake python3 ./platforms/js/build\_js.py build\_js @endcode ### Building the documentation with Docker To build the documentation `doxygen` needs to be installed. Create a file named `Dockerfile` with the following content: ``` FROM emscripten/emsdk:2.0.10 RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends doxygen \ && rm -rf /var/lib/apt/lists/* ``` Then we build the docker image and name it `opencv-js-doc` with the following command (that needs to be run only once): @code{.bash} docker build . -t opencv-js-doc @endcode Now run the build command again, this time using the new image and passing `--build_doc`: @code{.bash} docker run --rm -v $(pwd):/src -u $(id -u):$(id -g) "opencv-js-doc" emcmake python3 ./platforms/js/build\_js.py build\_js --build\_doc @endcode ## [Js Table Of Contents Setup](https://docharvest.github.io/docs/opencv5/js_tutorials/js_setup/js_table_of_contents_setup/) Contents opencv5 Js Table Of Contents Setup OpenCV 5 Js Table Of Contents Setup # Introduction to OpenCV.js {#tutorial\_js\_table\_of\_contents\_setup} - @subpage tutorial\_js\_intro Introduction of OpenCV.js and Tutorials - @subpage tutorial\_js\_usage Get started with OpenCV.js - @subpage tutorial\_js\_setup Build OpenCV.js from source - @subpage tutorial\_js\_nodejs Using OpenCV.js In Node.js ## [Js Usage](https://docharvest.github.io/docs/opencv5/js_tutorials/js_setup/js_usage/js_usage/) Contents opencv5 Js Usage OpenCV 5 Js Usage # Using OpenCV.js {#tutorial\_js\_usage} ## Steps In this tutorial, you will learn how to include and start to use `opencv.js` inside a web page. You can get a copy of `opencv.js` from `opencv-{VERSION_NUMBER}-docs.zip` in each [release](https://github.com/opencv/opencv/releases), or simply download the prebuilt script from the online documentations at "[https://docs.opencv.org/{VERSION\_NUMBER}/opencv.js](https://docs.opencv.org/%7BVERSION_NUMBER%7D/opencv.js)" (For example, [https://docs.opencv.org/5.0.0/opencv.js](https://docs.opencv.org/5.0.0/opencv.js). Use `5.x` if you want the latest build). You can also build your own copy by following the tutorial @ref tutorial\_js\_setup. ### Create a web page First, let's create a simple web page that is able to upload an image. @code{.js} Hello OpenCV.js ## Hello OpenCV.js imageSrc @endcode To run this web page, copy the content above and save to a local index.html file. To run it, open it using your web browser. @note It is a better practice to use a local web server to host the index.html. ### Include OpenCV.js Set the URL of `opencv.js` to `src` attribute of