Concerns about his health began over the summer and grew this month when he visited a Tokyo hospital two weeks in a row for unspecified check-ups.
He has acknowledged having ulcerative colitis since he was a teenager and has said the condition was controlled with treatment.
Mr Abe, whose term ends in September next year, is expected to stay on until a new party leader is elected and formally approved by parliament.
He said on Friday: “It is gut wrenching to have to leave my job before accomplishing my goals.”
In a country once known for its short-tenured premiers, his departure will mark the end of an unusual era of stability that saw him strike up strong ties with President Donald Trump even as Mr Abe’s ultra-nationalism riled the Koreas and China.
While he pulled Japan out of recession, the economy has been battered by the coronavirus pandemic, and he has failed to achieve his cherished goal to formally rewrite the US-drafted pacifist constitution because of poor public support.
Mr Abe is a political blue blood who was groomed to follow in the footsteps of his grandfather, former prime minister Nobusuke Kishi, and sought to make Japan a “normal” and “beautiful” nation with a stronger military and more focus on the non-political emperor.
He abruptly resigned from his first stint in office in 2007 due to his health, which had fuelled concerns about his recent condition.
On Monday he became Japan’s longest serving prime minister by consecutive days in office, eclipsing the record of Eisaku Sato, his great-uncle, who served 2,798 days from 1964 to 1972.
Shigeru Ishiba, a hawkish 63-year-old former defence minister and Mr Abe’s arch rival, is a favourite next leader in media surveys, although he is less popular within the ruling party.
Low-key former foreign minister Fumio Kishida, defence minister Taro Kono, chief cabinet secretary Yoshihide Suga, and economic revitalisation minister Yasutoshi Nishimura, who is in charge of coronavirus measures, are widely tipped in Japanese media as potential successors.
The end of his scandal-laden first stint as prime minister was the beginning of six years of annual leadership change, remembered as an era of “revolving door” politics that lacked stability and long-term policies.
When he returned to office in 2012, Mr Abe vowed to revitalise the nation and get its economy out of its deflationary doldrums with his “Abenomics” formula, which combines fiscal stimulus, monetary easing and structural reforms.
