When I started to learn Japanese I wanted to listen to anything and everything. This was good, but it was just too difficult. I became frustrated.
Then, I wanted to learn by listening to small selections of Japanese again and again and again. This was also good, but natural conversation is not like that.
I will write a few tips here about how to listen better.
Practice Making Predictions
Before you listen to someone or something try to imagine what they are going to say. This is important. If you are not successful, that is OK. You can learn from your mistakes. You must continue until you can do this (make predictions) better.
You can practice with tv shows (get free tv off the internet). Watch the dramas, and the Sit coms. Reality tv is also good, but it might be a little difficult.
You can practice with podcasts. Each podcast has a theme, and many have daily topics. Use those themes and topics to help you make predictions about what they are going to say. When they say something different, write it down and memorize it. You might learn some new expressions.