Why did Father Gilligan grieve? Why did he repent?

The details you need to answer your question are in the
second and third stanzas of the poem.  Here's the first four stanzas, to give you the
full context, of "The Ballad of Father Gilligan," by Yeats.  They include the details
you ask about:

The old priest Peter Gilligan
Was
weary night and day;
For half his flock were in their beds,
Or under
green sods lay.

Once, while he nodded on a chair,
At the
moth-hour of eve,
Another poor man sent for him,
And
he began to grieve.

‘I have no rest,
nor joy, nor peace,
For people die and
die’;
And after cried he, ‘God forgive!
My
body spake, not I!’

He knelt, and leaning on the
chair
He prayed and fell asleep;
And the moth-hour went from the
fields,
And stars began to peep.

I've emboldened
the lines that include what the priest sees as his grievous sin, and italicized his
repentence.  Actually, his reaction is probably quite natural.  Members of his flock are
dying faster than he can deliver the Last Rites to them.  He is dozing off in his chair
when he gets summoned by yet still another dying parishioner.  He is overworked and
exhausted, but when he reacts to the summons in a very human way, he sees his words and
behavior as not fitting for a priest.  He grieves and explodes, figuratively, and then
is immediatley sorry for what he says right after he says
it.

God, in the poem, though, seems to understand.  While
the priest is praying, begging for forgiveness, he falls asleep and misses the man's
dying moments, but God sends an angel in the priest's place to administer the Last
Rites.