That's so cute! You should totally do it.

The hardest part of this is in dataset creation. It's hard to clean and annotate the data and can be quite manual. That's why companies with lots of data will win.

There are automated techniques to help with segmentation, bandpass filtering, transcriptions, etc., but they're far from perfect.