There were two Route 32A's (or as William Safire might insist, "two Routes 32A.")
Windham
The 1942 state highway log listed Route 32A as a 1.85-mile route in Windham, without further details. The ConnDOT spreadsheet mentions only that it was deleted in 1943.
My guess as to where Route 32A existed: South Windham Road, leading from Machine Shop Hill Road to Windham Road. The length matches (to about 0.1 mile); and newspaper articles from the 1940s mention that Route 32 was reconstructed here around 1940. Windham Road, to the east, would be the new Route 32, and South Windham Road (Route 32A) would be the old. I don't know if it was ever signposted.
New London - Waterford
In New London and Waterford, Route 32A was applied to a 3.19-mile stretch of Williams Street and Old Norwich Road. As with the Windham Route 32A, this route was previously part of Route 32. In the late 1930s, work began on a bypass of this stretch of Route 32, notorious for steep hills and narrow, sharp curves. In summer 1939, two of the four lanes on Mohegan Avenue, the new Route 32, opened to traffic. In 1940 work was complete; Mohegan Avenue became the new Route 32, and Route 32A was assigned to the old route.
Route 32A here was cancelled in 1943.