Holden's first starring role was in Golden Boy (1939), in which he played a violinist turned boxer. That was followed by the role of George Gibbs in the film adaptation of Our Town.
 
 With Gloria Swanson in Sunset Boulevard (1950)

After Columbia Pictures picked up half of his contract, he alternated between starring in several minor pictures for Paramount and Columbia before serving as a 2nd lieutenant in the United States Army Air Forces during World War II, where he acted in training films for the First Motion Picture Unit. Beginning in 1950, his career took off when Billy Wilder tapped him to star as the down-at-the-heels screenwriter Joe Gillis, who is taken in by faded silent-screen star Norma Desmond (Gloria Swanson) in Sunset Boulevard, for which Holden earned his first Best Actor Oscar nomination.

Following this breakthrough film, his career quickly grew in stature as he played a series of roles that combined good looks with cynical detachment, including a prisoner-of-war entrepreneur in Stalag 17 (1953), for which he won the Academy Award for Best Actor, a pressured young engineer/family man in Executive Suite (1954), an acerbic stage director in The Country Girl (1954), a conflicted jet pilot in the Korean War film The Bridges at Toko-Ri (1954), a carefree playboy in Sabrina (1954), a wandering college football star in Picnic (1955), a dashing war correspondent in Love Is a Many-Splendored Thing (1955), his most widely recognized role as an ill-fated prisoner in The Bridge on the River Kwai (1957), a World War II tug boat captain in The Key (1958), and an American Civil War military surgeon in The Horse Soldiers (1959), which also starred John Wayne.

He also played a number of sunnier roles in light comedy, such as the handsome architect pursuing virginal Maggie McNamara in the controversial Production Code-breaking The Moon Is Blue (1953), as Judy Holliday's tutor in Born Yesterday (1950), as a playwright captivated by Ginger Rogers' character in Forever Female (1953) and as Humphrey Bogart's younger brother, a playboy, in Sabrina (1954), which also starred Audrey Hepburn. In 1954, Holden was featured on the cover of Life.

His career peaked in 1957 with the enormous success of The Bridge on the River Kwai, but Holden spent the next several years starring in a number of films that rarely succeeded commercially or critically. By the mid-1960s, the quality of his roles and films had noticeably diminished.