Hersh was born to a Jewish family in Kovno, the capital of independent Lithuania. Hersh's father was a mechanic in a textile factory, and his mother had worked as a hat designer until he was born. The Gordons lived on the first floor of Hersh's grandfather's apartment building. Eight-year-old Hersh was in the third grade at public elementary school.
1933-39: In the summers my mother and I would go to my Aunt Ettel and Uncle Abraham in a small town not far away. We'd take a boat down the river to get there. From May to September they operated a small restaurant and motel, and my mother would help out with the cooking. My father would come out on the weekends. I talked my uncle into buying a ping-pong table and used to win money playing table tennis with the tourists.
1940-44: I was almost 16 when the Germans occupied Kovno and forced all the Jews into a fenced-off ghetto. One day the door of our ghetto apartment flew open and two Germans with guns told us to run to a nearby bridge. As hundreds of Jews from the ghetto walked along the bridge, people began saying prayers and wishing each other goodbye. Germans with heavy rubber clubs started beating everyone over the head. Then they sorted us out as if we were animals, inspecting us to see how fit we were. I was put with others my age.