I never received my bill or I received it late, do I still have to pay the penalty?

Yes.  Failure to mail a statement or the failure of the property owner to receive a statement will not affect the validity of the taxes or penalty as dictated by law (New York State Real Property Tax Law §922).  The Tax Collector does not have the legal authority to waive statutory penalty charges, nor does any other official.  These regulations are governed by the Real Property Tax Law.  If the collector waives any penalty he/she will be held personally responsible.  All records are audited by State examiners and there is no discretion in this matter.