Inlaws....Is this right????

ok so here's the thing my boyfriend/father of my 3 year old daughter's parents are horrible.(background info) They made my boyfriend drop out of school at 16 so he could work and pay their whole full rent.(Today) well ever since we've had our daughter they have never once came and seen her, not once. We brought over to their house on a number of occasions, against my better judgement. but i will not bring her to them ever again. They smoke weed in the house, and the place is disgusting!!! They have pitbulls running around w/ mange and fleas. His mom has never cleaned a day in her life. My boyfriend still (when im in college) takes my daughter over to that house in spite of what i say. I know that those are his parents but i feel like if they cared then they would once in a while call their son or granddaughter just to say hi. Or once go to their house. They also go out of their way to tell people that i keep my daughter from them.