On the lookout for a new architecture, people told me to use SAGAN. The results are – interesting.
I wasn’t satisfied with a normal GAN architecture – so I started using SAGAN. So I set up my architecture, loaded in my data and got busy.
The start wasn’t satisfying – I encountered mode collapse, dark spots in my designs, floating heads – but the results were different, it seemed like they weren’t as randomly put together as my previous one, as if they’d pay Attention to what was going wrong and worked on that.
First result after running with SAGAN as per @DrBeef_‘s suggestions. Tried multiple losses to prevent mode collapse.
🙋♂️ Designs are getting more diverse, which is great
🌊 Output seems a little smoother
🎭 A little bit more noise, see background (input data only has grey bg) https://t.co/xDlwMt9j1C
— Dominic Monn (@dqmonn) December 29, 2018
However, at some point, the progress stopped. The designs just didn’t get better, and over time, they got darker. The faces slowly became unclear, the bodies started flowing away – until all that was left were nightmare creatures.
While I think SAGAN could have potential to become something great, it seems to suffer a lot more from the usual GAN problems, and I need to pay closer attention to the parameters, training time and data.