toggleFade on top of div only

toggleFade on top of div only

I have a div that when I click on it it toggles another div right
underneath it and when you click it again it goes away. The problem is,
there are input forms in the div that shows up and when I click ANYWHERE
in the div that appears it disappears again. Here is my code:
$(document).ready(function(){
$(".click").click(function(){
$(".fade", this).fadeToggle();
});
});
<div class="click">
<div style="height:40px; width:900px; border-radius:5px; cursor:hand;
cursor:pointer;"><img src="images/editloadout.png" /></div>
<div class="fade">
</div>
</div>
Is there a way I can make it so that it toggles only when you click the
editloadout.png picture?