There are two entries for Hollenbeck in the U.S. Civil War Draft Registrations Records, 1863-1865. The first lists his name as Joseph Hollenback, age 23, single, living at the northeast corner of Third Street and Green in St. Louis, born in New York and employed as a constable. While there is no date for the record, it gives the age of each person listed as of July 1, 1863. The second record is dated August 31, 1863, and has the exact same information as the first.
John Maurath, at the Missouri Civil War Museum, sent me some information awhile back stating that he found a Joseph Hollenback enlisted in Captain Loeblein's Company A, 10th Regiment, Enrolled Missouri Militia. According to the record, this Hollenback enrolled with the unit on July 31, 1864. Without any real evidence, I think this is our guy. The EMM was, essentially, a reserve force that was called up only when needed. From what little I know about it, Hollenbeck, like all males of service age, was forced to enroll in the EMM and I see no evidence that he actively took part in any engagements. Even as a member of the EMM, he would have been in St. Louis in September of 1864 when, according to the records I found, Hollenbeck died.
So, while, Hollenbeck signed up for the draft and enrolled in the EMM, I don't believe that he was ever a member of the active duty military during the Civil War.