Average Column Depending On Values In Another Column

I have tried a few other examples in previous threads that use COUNTIF and ISERROR but I just can't get it right.

Basically I want to average the cells in column J (J2:J6,J8:J12,J14:J18, etc).
My aim here is to determine the average value for each day (Averaging 0 values in this instance is OK).

Now, the date in column A (A2:A6,A8:A12,A14:A18, etc) sometimes returns a blank value, depending on the day of the week (see attached example). This is because, in the full version of the workbook, I have Column A date linked to a perpetual calendar and therefore the dates can change cells depending on the year.

For the Date Cells that return a blank value, I do not want the AVERAGE equation to include the 0 value on these days.

For an example I have attached a month of February where there are blank date cells at either end of the column.