I had this problem with view panes in panels. I create a view pane, add it to panel and it works nicely when there is data. But when there is not I got just some empty html data, problem is that this html data was wrapper data and it showed some CSS elements even when there was no data. So some lines or arrows or something would appear, or when I used jCarousel like here I got title and arrows http://drupal.org/node/1912218#comment-7044226
So what to do about this. Problem is the way of thinking about this. It is not the panels is not working or bad or buggy, it is that you miss the point. Panels are showing this data as there is a data. For example the above problem with jCarousel:
When you have a view pane, you are usually checking nodes, so lets say you are displaying a jCarousel and the data source is a image field in your content type that also has body and title. So when you do this and create a panel pane, it finds out that there is a result, it is a node, just without data, but you will get a HTML result.
So then what I thought of is this. Instead of using some custom stuff. I just did this. I added a views filter (view pane)
Content: Images:delta (not empty) which basicly checks if there is some images in this gallery, if not views will not display any results, and this WORKS.
To have much more control over Panels and things you want to appear or not you can make a Visibility Rule in Panels.
1. Using the PHP Code and getting the $contexts variable, that has all the node data (if you are overriding node foe example) so you can check upon what is present and what are the value and act accordingly, providing in the end "return true/false" to set visibility of Pane.
2. Use panels predefined fields checks (for example check on the value of some field) or add a context in panel, for example taxonomy reference and you can get a taxonomy field to check on value, so you can hide a pane if a node has or doesnt have a particular term, this is pretty powerful.