The first time in the Torah that God speaks to any living creature, the speech is directed at fish. In Genesis 1:22
”And God blessed them saying: Be fruitful and multiply, and fill the waters in the seas…”
 
The fish symbol is also connected to the Sabbath - the 7th day- as the numerical value of the word fish equals the number seven. As such, on the Sabbath it is traditional to eat fish.
On Rosh HaShana it is also customary to place on the festive table the head of the fish, symbolizing the wish to be at the "head" of the community in righteousness.