Inpainting, also known as "data completion" or "disocclusion" is the process of restoring missing or corrupted parts of an image or video in a way so as to make the results look visually pluasible and natural to human eyes (see Figure 1). The problem of inpainting is encountered in various image processing applications such as image and video restoration, editing (e.g., object removal and removal of scratches from old photographs), disocclusion in image based rendering, interpolation, loss concealment in data transmisson, and texture synthesis or image resizing (e.g., enlargement).

Figure 1: Image inpainting problem. Left: The original image, Middle: input image with region to be inpainted indicated in gray color, Right: inpainting result.

Inpainting is an ill-posed inverse problem and does not have a well defined unique solution. To solve this problem, it is therefore necessary to introduce image priors. Currently the developed algorithms in the literature are guided by the assumption that pixels in the known and unknown parts of the image share the same statistical or geometrical properties. This assumption translates into different local or global priors, with the goal of having an inpainted image as physically and as visually pleasing as possible to the human eyes.

Inpainting problems promote two main objectives, 1) propagate the intensity information smoothly along the isophote diffusion direction of the intensity from external to internal missing regions, 2) select from the background image or generate suitable texture to fill in the missing regions. Based on these objectives inpainting algorithms can be roughly classified into two different catogeries: diffusion-based inpainting and examplar-based inpainting. Diffusion-based algorithms deals with the propagation of local information with smoothness priors and thus does well to restore small and narrow gaps in the image. They can be formalized using partial differential equations (PDE) and thus use PDE-based regularization to solve the problem. Examplar-based algorithms on the other hand aim to better recover the underlying texture of the missing area. These algorithms rely on the locality and stationarity of the pixels and thus solve the inpainting problem by sampling and copying a patch of pixels from a sample texture that best matches the known neighborhood of the the input region. Examplar-based algorithms generally provide good results for large object removal but may fail to recover some structures (e.g., lines, curves).

In this project, we merged empirical properties of diffusion-based inpainting and examplar-based inpainting to produce a robust and efficient system capable of obtaining pluasible inpainting results both geometrically and texturally.