No, there’s no drawbacks to ever using Custom HTML or to never use Blank Panels. Whenever I think that later I maybe will edit the HTML, I just use a Custom HTML, even if I never edit the html or style.
However, I can’t see why I can’t change the type on the go on the editor. Now, with the new Designer it’s even “harder” to convert, since the “draw to select multiple objects” is gone and I prefer to don’t have custom html if I don’t intend to use it. It’s harder to detect and the “Find” feature don’t search in those parts of the forms, so it’s harder to find styles that may work in unintend ways.