It is easier to give to people if you know them, and it is easier to know people if you talk to them or spend time where they are. I rarely hand out cash to panhandlers on the streets of my town, but I want to look at them and speak to them if I can. Not all are interested: I may be interfering with panhandling.
But giving is something that I need to do for me, not just for the recipients. So I give to groups that help to meet the needs of the poor and homeless. If someone tells me that they are hungry or in need, I ask if they know about this or that source of help. It's not perfect, but it's a way.