My recent research are mainly on multimedia processing that cover the following topics:


AVE (Automatic Video Editing)

Introduction

Many personal video camera owners accumulate a large collection of videos documenting important personal or family events. Despite their sentimental value, these videos are too tedious to watch because (1) many home videos are comprised of extended periods of inactivity or uninteresting activity, with a small amount of interesting video, and (2) the poor sound quality of many home videos exacerbates the associated tedium. Therefore, it is desirable to condense large amounts of uninteresting video into a short video summary. Unfortunately, the sophistication of editing tools make it difficult to use for the average home video producer. Further, even simplified editing tools require users to select and arrange the portions of video of interest. The time and effort required to provide the creative input necessary to produce a professional looking video summary discourages the average home video producer.

This project is to provide a convenient way for users to edit and summarized video content. Our system merely requires users to input their video content and select some music, then it automatically generates a professional looking video production. Users can also input photos with music to create a music slideshow production.

Demo: music slideshow (1.31MB), music video (0.95MB)
(To save network bandwidth, the demo files are saved with lower quality.)


A home video case


NPR (Non-Photorealistic Rendering)

Introduction

Beauty is subjective rather than objective, it relies on how it is being delivered and how the observer perceives. Photorealism used to be the only goal for computer graphic scientists, where simulating realism as much as they could was the only way out. However, many imageries generated by artists are not readily apparent in photographs or real life, whereas the observer can still comprehend the information that artists are trying to deliver. When using images to communicate the essence of a scene, simulating reality is not as important as creating the illusion of reality. Non-photorealistic rendering (NPR) technique is a technique that makes imagery look as though it was made by artists. It brings together both art and science, concentrating less on the process and more on the communication content of an image. NPR involves stylization and communication, usually driven by human perception. Knowledge and techniques long used by artists are now being applied to computer graphics to emphasize specific features of a scene, expose subtle attributes, and omit extraneous information to give rise to a new field.

3D NPR: Stylized Outline

Stylized Outline effect is to render the object's outlines, including silhouettes, surface boundaries, and creases, with special style, such as pencil, crayon, etc.
*The method was applied for U.S./Taiwan patent. (Appl. No. 20040085314)

3D NPR: Cartoon Shader

The Cartoon Shader is to render a 3D object with cartoon style.

3D NPR: Colorful Shader
Image-based NPR: Contour Drawing

The "Contour Drawing" effect is one of NPR effects that transfers source image into "Contour Drawing" style.

Image-based NPR: Mosaic Art

The "Mosaic Art" effect is one of NPR effects that transfers source image into "Mosaic Art" style. We use a new algorithm to create better mosaic reproduction, where tiles are aligned to emphasize edges, and different sizes and shapes of tiles are used to create varieties.

Image-based NPR: Impressionist

The "Impressionist" effect is expressed by broad, vigorous, brilliant color, and swirling brush-strokes to create "Impressionistic" style paintings.

Image-based NPR: Comic

The "Cartoon" effect transfers source image into "Cartoon" style. Cartoon rendering consists of two processes: inking and painting. Inking process gives the image clear drawing of outlines and feature lines, while painting process fills simplified, smooth or flat colors on each area of the image.




Particle System

Introduction

A particle system can create 2D/3D visual effects, such as snow, smoke, fire, and so on. Each type of visual effects is generated by a group of moving particles, which are emitted by an emitter. Here we quote description from Jeff Lander: "A particle system is basically just a collection of 3D points in space. Unlike standard geometry objects, particles making up the system are not static. They go through a complete life cycle. Particles are born, change over time, and then die off. By adjusting the parameters that influence this life cycle, you can create different types of effects."

2D Particle System

This fuction is to perform particle animation effect on photos, for examples, to create the phenomenon of falling snow or flying bubbles.

3D Particle System

The system can create 3D visual effects, such as snow, smoke, fire, or customized effects. Each type of visual effects is generated by a group of moving particles, which are emitted by an emitter. User can control the motion behavior, as well as the shape and color of particles, or control the transformation of emitter, to create interactive effects with scene objects.




Others


Video Transition Effect: Album-3D TX

This function is a transition effect that makes a transition from one video clip (Va) to another (Vb) by simulating the way we browse through an album. The album is a 3D rendered object where Va and Vb are put together as photos or pictures.

Graphics: Render Moon on a Photo

The purpose of this function is to render a moon in image. You can customize the appearance of moon or put it anywhere of image to impress your scene with romance, wonderment, fantasy, and tranquility.

Image Filter: Edge-Preserving Smoothing

This function is to perform edge-preserved smoothing. It is similar to a regular blur, except that neighboring pixels that differ more than the given threshold will not be blended with. This way an image can be smoothed out without losing details. The smoothing algorithm could be chosen as uniform, triangle, or Gaussian blur.

Image Segmentation