A few lessons learned with this little project:
1) You can't be lazy with clean up. All the shapes need to stay consistent in size and volume and need to respect their own arcs. So for clean up I think it's best to also clean up keys, then breakdown and then inbetweens.
2) Pencil tool for clean up is really great for super quick coloring. For my next animation I'll use the brush with some texture and see if there are any benefits to using it over the pencil tool.
I would've liked to spend some time doing a background and doing a proper shading job, but just wrapped this up pretty simple using a copy of the color fills to generate an automated highlight.
You can also see a video that goes through the entire process from start to finish on youtube, hope this is useful to somebody out there! :)