In 1969, Joe was selected by the Pittsburgh Steelers with the fourth pick of the NFL draft and spent his entire career with them until his retirement in 1981. When Greene was drafted, a newspaper headline asked, Who’s Joe Greene? The question was quickly answered as Greene became so good that teams double-teamed, and even triple-teamed, him throughout his entire career. In addition to his skills, other teams saw Greene as a threat because of his size.

After he was drafted, Greene quickly established himself as a dominant defensive player. He was strong, quick and intense. He was the NFL’s Rookie of the Year in 1969, even though he played on a Steelers team that went 1–13 in Chuck Noll’s first year as its head coach. He had no tolerance for losing, and the team veterans quickly took notice. His intense desire to win rallied the veterans around him, and with great drafts as well as superb coaching, the Steelers franchise soon began to undergo a dramatic makeover. Joe Greene was credited as the cornerstone of the great Steelers dynasty and the most important player in Steeler’s history.

Greene was the leader and the anchor of the “Steel Curtain” defense that won four Super Bowls in the 1970s. He was recognized as the NFL Defensive Player of the Year in both 1972 and 1974. He, along with other members of the Steelers’ front four (L. C. Greenwood, Dwight White and Ernie Holmes) even appeared on the cover of Time magazine. In Super Bowl IX, Greene became the first player ever to record an interception, a forced fumble, and fumble recovery in a single Super Bowl. He went to the Pro Bowl 10 times during his career.

Greene is also well known for the “stunt 4–3” defense, in which he would line up at an angle, between the center and guard, and would explode into the line taking up 2–3 blockers. He started doing this sometime in the 1974 season, and while it cut down on the number of sacks he racked up, it freed up his other defensive teammates like middle linebacker Jack Lambert to make tackles with ease.

After leading the Steelers to another Super Bowl win after the 1975 season over the Dallas Cowboys in Super Bowl X, Greene missed the first several games of the 1976 season with a back injury. The Steelers started off the season 1–4 and looked like they would not make the playoffs. Quarterback Terry Bradshaw was also injured and was replaced by rookie Mike Kruczek. The season looked lost. But Greene and the Steelers defense carried the Steelers to nine straight wins and the playoffs. With a defense considered one of the best in NFL history, the 1976 Steelers held opponents to an average of less than 10 points per game (138 points over 14 games). During their nine-game winning streak, the Steelers defense recorded five shutouts, another modern record, and gave up a total of just 28 points (roughly 3 points per game). The defense allowed only two touchdowns over nine games.

Ten of the eleven starters on that 1976 Steelers team were players who made the Pro Bowl at least once in their career (eight starters made the Pro Bowl after the 1976 season). Middle linebacker Jack Lambert along with Greene, became the emotional leaders of the defensive squad. Greene continued to perform at an all-pro level, becoming a 5-time All-Pro (1972–74, 77, 79) and in 1969 receiving the first of his 10 Pro Bowl invitations. Greene retired after the 1981 season after 13 years in the league.

He played in 181 games, had 78.5 sacks (unofficially, as sacks were not an official statistic until 1982) and 16 fumble recoveries.