The history of SMS group is inextricably linked to the Weiss family of entrepreneurs. "Founding father" Carl Eberhard Weiss (1841–1904) was born in Obertürkheim near Stuttgart. Having completed his blacksmith training and after some time traveling, he started work as a forgeman in the machine-building workshop at A. & H. Oechelhäuser in Siegen. Five years later, Carl Eberhard Weiss established his own small forging business with three employees in 1871. Carl was a dedicated entrepreneur who steadily expanded his product range and market position.

Once they had finished their training as forgemen, his sons Carl (1872–1946) and Ernst Heinrich Weiss (1875–1932) joined their father's business from 1896, after which they completed their studies in mechanical engineering and a commercial apprenticeship respectively. They expanded the family-owned business by acquiring other mechanical engineering companies, then merging them to create Siegener Maschinenbau AG in 1918. With the acquisition of Maschinenbau AG vorm. Gebr. Klein in Dahlbruch in 1927, Carl and Ernst Heinrich Weiss expanded their production portfolio to include the manufacture of rolling mills for the steel and nonferrous metals industry. In 1939, the abbreviation "Siemag" appeared in the company's name for the first time, although it had been used colloquially up to that point.

In 1928, Bernhard Weiss (1904–1973), son of Ernst Heinrich Weiss, joined the company as an authorized signatory and company representative. When his father died in 1932, he took up the position Ernst Heinrich had held, and was appointed to the board of management. By 1940, he had achieved his corporate objective of holding all shares in the company, after which he led the business as its sole owner. After the Second World War, he expanded Siemag, making it a leading German rolling mill manufacturer. By the end of the 1960s, Bernhard Weiss had already recognized the need to join forces with a major market player in order to remain competitive internationally.

In 1971, Heinrich Weiss (born 1942) became chairman of the managing board of Siemag. In 1968, he took over as managing director of Siemag Transplan GmbH in Netphen. Following the merger between Schloemann and Siemag to form Schloemann-Siemag AG, Heinrich Weiss became its CEO and chairman in 1974. M.A.N. held 51 percent of the shares in Schloemann-Siemag AG, to which the short name SMS was added in 1980, with the family holding company Siemag Weiss KG owning 49 percent of the shares. Both partners each held half of the voting capital. In the years that followed, Heinrich Weiss expanded the company to create the world's largest manufacturer of plant and machinery for the iron and steel sector. With the repurchase of the remaining shares held by M.A.N. in 2007, Heinrich Weiss once again brought SMS group, which by then had grown considerably, back into the ownership of the Weiss family. On July 1, 2013, Heinrich Weiss moved from his position on the managing board of SMS group and took over as chairman of the supervisory board. In 2016, Edwin Eichler was named as the new chairman of the supervisory board, while Heinrich Weiss, his predecessor, continued to be a member of the supervisory board.