Solved OpenAI Gym CartPole v0 in 2016/09/07. I had read Andrej Karpathy’s implementation (nearest neighbor) and decided that I wanted to solve it in a different way. Then I started to study reinforcement learning and implemented deep Q-network.
Solved OpenAI Gym CartPole v1 in 2016/09/14. Used the solver of v0 (DQN) with tweaked parameters.
Pet Projects (in my free time)
In 2010, I was interested in images. So I implemented an image viewer in “mathematical way”. It focused on processing image with simple math operators like pixel-wise scaling, images sum, etc. One of my goals was to make it available on both Windows and OSX so it’s implemented with wxWidgets.
High Dynamic Range Imaging
- Implement Paul Debevec’s algorithm (SIGGRAPH 97).
- c/c++, libpng, Numerical Recipes in C (SVD).
Gradient Domain High Dynamic Range Compression
- Raanan Fattal’s algorithm.
- Multigrid algorithm was implemented to solve the final Poisson equation.
- c/c++, numerical analysis.
Scale Invariant Feature Transform - SIFT
- Direct Linear Transform
Faked High Dynamic Lighting
- c/c++, d3d9, hlsl, Win32 screen saver.
- Kawase’s bloom filter.
- c/c++, d3d10
- 128 point lights around each ball.
Software 3D Rendering
- Both the geometry and texture are from Warcraft III.
San Angeles Observation in WPF
- c#, WPF.
Image Reaper and Chrome Extension
- Link chrome extension and python scripts.
- c/c++, ddraw, dshow, dsound.
- My first programming project, in 2003.
- Sprites belongs to Blizzard’s StarCraft.