Deep sketching : vectorization of sketched drawings using deep learning

Abstract

Sketch vectorization is an essential step to bridge the gap from hand drawn rough sketches to images that can be interpreted by computer based systems. Through this dissertation, a Fully Convolutional Neural Network is designed to automatically clean raster rough sketches into their line drawing counterpart. A custom loss function was created inspired from traditional feature based methods of analysing the quality of line drawings. The loss function quantifies the quality of the lines and gaps being extracted. It also promotes that the lines extracted are clearly separate from the background. A dataset was curated from images found ‘in the wild’. The ground truth images were drawn by hand and method to align hand drawn ground truths to the found sketches, irrespective of printing and scanning resolution, was created. The curated dataset was used along with another dataset created in similar conditions. The design proposed and the dataset used allowed sketches from different sources to be simplified. Over strokes were converted to their clean line counterparts, shading and hatching, which are artifacts which hinder the vectorization process which follows, are removed whilst detail from the sketches was retained. A vectorization algorithm is then applied to the output from the cleaned line drawing. The time taken to convert the sketched image to its cleaned vector counter parts is quick, hence the method can be used in down stream applications. Finally, metrics measured show an improvement of the proposed loss function when compared to the Mean Squared Error (MSE) loss function as well as an improvement upon the more simplified version of proposed loss functions. For validation the output is also compared to other varieties of methods proposed which are deep learning based or feature based.