On 13 June 1923, Captain E. J. King, Commander, Submarine Division Three (later Fleet Admiral and Commander in Chief, U.S. Fleet, during WW II), suggested to the Secretary of the Navy via the Bureau of Navigation (now known as BuPers) that a distinguishing device for qualified submariners be adopted. He submitted a pen-and-ink sketch of his own showing a shield mounted on the beam ends of a submarine, with dolphins forward of, and abaft, the conning tower. The suggestion was strongly endorsed by Commander Submarine Division Atlantic. Over the next several months the Bureau of Navigation solicited additional designs from several sources. Some combined a submarine with a shark motif. Others showed submarines and dolphins, and still others used a shield design. A Philadelphia firm, Bailey, Banks & Biddle (BB&B), which had done work in the field of Naval crests, was approached by the Bureau of Navigation with the request that it design a suitable badge. Two designs were submitted by the firm, one of which was the 1926 class crest from the Naval Academy. These two designs were combined into a single concept. It was a starboard angle on the bow view of an "O" class submarine, proceeding on the surface, with bow planes rigged for diving, flanked by dolphins in a horizontal position with their heads resting on the upper edge of the bow planes. On 20 March 1924, the Chief of the Bureau of Navigation recommended to the Secretary of the Navy that the design be adopted. The recommendation was accepted by Theodore Roosevelt, Jr., Acting Secretary of the Navy. His acceptance is dated March 1924. Today the original BB&B design is used by many manufactures without modification, while others choose to take some artistic license, especially in updating the class of submarine depicted. 

The submarine insignia was to be worn at all times by officers and men qualified in submarine duty attached to submarine units or organizations, ashore and afloat, and not to be worn when not attached. In 1941 the Uniform Regulations were modified to permit officers and men qualified who were eligible to wear the submarine insignia after they had been assigned to other duties in the naval service, unless such right had been revoked.

The officers' insignia was a gold metal pin (gold plating over "sterling" silver, or bronze), worn centered above the left breast pocket and above the ribbons and medals. Enlisted men wore the insignia, embroidered in silk, white silk for blue clothing and blue silk for white clothing. This was sewn on the outside of the right sleeve, midway between the wrist and elbow. The device was two and three-quarters inches long.
 
In 1943, the Uniform Regulations were modified to provide that:  "Enlisted men, who are qualified and subsequently promoted to commissioned or warrant ranks, may wear enlisted submarine insignia on the left breast until they qualify as submarine officers, at which time this insignia would be replaced by the officers' submarine pin." 

In mid-1947, the embroidered device shifted from the sleeve of the enlisted men's jumper to above the left breast pocket. On 21 September 1950, a change to the Uniform Regulations authorized officers the option of either a gold bullion embroidered sew-on insignia, or gold plated pin-on insignia.  Enlisted submariners were given the option of either a silver bullion embroidered sew-on insignia, or a "sterling" silver (or silver plated), metal pin-on insignia for the dress uniform. This was in addition to the silk embroidered  insignia sewn on the undress uniform.