OpenAI Gym

OpenAI Gym CartPole v0 Solved OpenAI Gym CartPole v0 in 2016/09/07. I had read Andrej Karpathy’s implementation (nearest neighbor) and decided that I wanted to solve it in a different way. Then I started to study reinforcement learning and implemented deep Q-network.

OpenAI Gym CartPole v1 Solved OpenAI Gym CartPole v1 in 2016/09/14. Used the solver of v0 (DQN) with tweaked parameters.



Pet Projects (in my free time)

Image Viewer

image_viewer_0 image_viewer_1

In 2010, I was interested in images. So I implemented an image viewer in “mathematical way”. It focused on processing image with simple math operators like pixel-wise scaling, images sum, etc. One of my goals was to make it available on both Windows and OSX so it’s implemented with wxWidgets.

High Dynamic Range Imaging

hdri_tone_mapping hdri_radiance_map

  • Implement Paul Debevec’s algorithm (SIGGRAPH 97).
  • c/c++, libpng, Numerical Recipes in C (SVD).

Gradient Domain High Dynamic Range Compression


  • Raanan Fattal’s algorithm.
  • Multigrid algorithm was implemented to solve the final Poisson equation.
  • c/c++, numerical analysis.

Scale Invariant Feature Transform - SIFT


  • c/c++

Image Stitching

stiching stiching

  • c/c++
  • Direct Linear Transform

Faked High Dynamic Lighting

fake hdr

  • c/c++, d3d9, hlsl, Win32 screen saver.
  • Kawase’s bloom filter.

Deferred Lighting

deferred lighting

  • c/c++, d3d10
  • 128 point lights around each ball.

Software 3D Rendering

software rendering software rendering

  • c/c++.
  • Both the geometry and texture are from Warcraft III.

Path Tracing

path tracing path tracing

  • c/c++.

San Angeles Observation in WPF

san angeles san angeles

  • c#, WPF.

Image Reaper and Chrome Extension

  • python, beautifulsoup, httplib2, sqlit3, javascript.
  • Link chrome extension and python scripts.


4x4 4x4

SC Like

sc like

  • c/c++, ddraw, dshow, dsound.
  • My first programming project, in 2003.
  • Sprites belongs to Blizzard’s StarCraft.