*

upload_article_image

Melting 'frozen memories,' AI helps Japanese recall war days

When Tokuso Hamai saw the colorized version of an old black-and-white photo of a picnic held under cherry tree blossoms sometime before World War II, forgotten memories of family members, most of whom died in the atomic bombing of Hiroshima in 1945, came pouring out.

“In colorized photos, people come to life,” said Hamai, now 86. “I often played near (the picnic site), and sometimes I would do some naughty things and get scolded by my father.”

The power of a colorized photo to reignite lost memories was eye-opening for Anju Niwata, a student who gave Hamai the colorized photo as a present three years ago.

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and its original black and white U.S. Navy photo that the heavily listing USS Franklin, center, is provided assistance by USS Santa Fe after the aircraft carrier had been hit and set afire by a single Japanese dive bomber, during the Okinawa invasion, on March 19, 1945. Niwata and Watanave are adding color to pre-war and wartime photographs using a combination of methods. These include AI technologies, but also traditional methods to fill the gaps in automated coloring. These include going door to door interviewing survivors who track back childhood memories, and communicating on social media to gather information from a wider audience. The team has brought to life more than a thousand black-and-white photographs that illustrate the pre-war lives of ordinary people and chronicles the onset and destruction caused by World War II. (U.S. NavyAnju Niwata & Hidenori Watanave via AP)

The 75th anniversary of the end of World War II is Saturday, and Niwata, now 18, said she hopes it will bring attention to her project with a Tokyo University professor to painstakingly colorize photos using artificial intelligence and their own research to spark lost memories for the rapidly aging generation who experienced the war.

“Seeing Niwata share the colorized pictures with Hamai, and then watching him recall his old memories one after another, made it feel like the ice around his frozen memories was melting away,” said Hidenori Watanave, the professor who taught Niwata how to colorize monochrome pictures using AI.

Niwata and Watanave call their photo colorization project “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the war.

This photo combination shows digital colorization, left, by Anju Niwata and Hidenori Watanave, and its original black and white Kyodo photo that Japanese Emperor Hirohito, right, meets Gen. Douglas MacArthur at the U.S. Embassy in Tokyo on Sep. 27, 1945. Niwata and Watanave are adding color to pre-war and wartime photographs using a combination of methods. These include AI technologies, but also traditional methods to fill the gaps in automated coloring. These include going door to door interviewing survivors who track back childhood memories, and communicating on social media to gather information from a wider audience. The team has brought to life more than a thousand black-and-white photographs that illustrate the pre-war lives of ordinary people and chronicles the onset and destruction caused by World War II. (Kyodo NewsAnju Niwata & Hidenori Watanave via AP)

Watanave and Niwata use three different types of AI photo coloring software. The AI is useful in identifying the accurate colors of natural things, such as the sea, the sky and human skin, but it cannot accurately colorize human-made objects like roofs and clothes, Watanave said.

So Niwata and Watanave painstakingly finish the AI-colorized photos by hand to get more accurate colors based on the photo owners’ memories and advice from experts. They also look through historical documents and archives that show what the colors should look like.

Some photos take a few months to finish.

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and original black and white photo that U.S. Army and Coast Guardsmen stand at attention as the American flag is raised over Akashima, Japan on April 2, 1945, the little island, only a few miles from Okinawa. The flag is from a Coast Guard LST. Niwata and Watanave are adding color to pre-war and wartime photographs using a combination of methods. These include AI technologies, but also traditional methods to fill the gaps in automated coloring. These include going door to door interviewing survivors who track back childhood memories, and communicating on social media to gather information from a wider audience. The team has brought to life more than a thousand black-and-white photographs that illustrate the pre-war lives of ordinary people and chronicles the onset and destruction caused by World War II. (U.S. Coast GuardNavy RadiophotoAnju Niwata & Hidenori Watanave via AP)

For Watanave, Twitter has become a powerful platform to pursue the colorization project.

When he posted a picture of the Hiroshima atomic bomb mushroom cloud that the Al software had colorized as white, a film director suggested that it should be more orange.

Watanave checked the testimonies of those who saw the mushroom cloud and also researched the components of the atomic bomb to see if it could actually make an orangish color.

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and original black and white photo that the first of 20 Japanese emerges from an Iwo Jima cave on April 5, 1945, with his hands in the air. The group had been hiding for several days. Niwata and Watanave started the photo colorization project in 2018. They call it “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the Pacific War. (U.S. Army Signal CorpsAnju Niwata & Hidenori Watanave via AP)

After he confirmed that it could, Watanave added orange to the picture.

While the accuracy of the color is important, Niwata and Watanave said the most vital thing is that the colorized photos match the memories of the photo owners.

Time, however, is running out; the average age of the atomic bomb survivors is about 83.

This photo combination shows digital colorization, left, by Anju Niwata and Hidenori Watanave, and its original black and white Kyodo photo that three Japanese schoolgirls are seen weeping in front of Japanese Emperor Hirohito's palace on Imperial Plaza in Tokyo, in Aug. 15, 1945. Niwata and Watanave started the photo colorization project in 2018. They call it “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the Pacific War. (Kyodo NewsAnju Niwata & Hidenori Watanave via AP)

There are often moments of wonder when the elderly see the revitalized photos.

When Niwata showed the colorized version of a family photograph to a war survivor who had dementia, for example, he remembered the type of flowers in the photograph; just a few weeks later, he was unable to speak.

Niwata said that publishing the colorization book during the coronavirus outbreak has made her think about the pandemic's link to the war.

This photo combination shows digital coloring by Anju Niwata and Hidenori Watanave left, and original black and white AP photo that Col. Paul W. Tibbets, Jr., pilot of the Enola Gay, the plane that dropped the atomic bomb on Hiroshima, Japan, waves from his cockpit before takeoff from Tinian Island in Northern Marianas, Aug. 6, 1945. (Anju Niwata & Hidenori Watanave via AP)

“Our everyday lives have been stolen away by the coronavirus in a flash, which I think resembles what happened in the war. That’s why I feel like now is an opportunity for people to imagine (wartime life) as their own experience,” she said.

Watanave hopes that using new technology will help younger Japanese feel more of an attachment to those who lived through the war.

“People are forgetting wartime memories. We need to revitalize those old memories by using the latest method of expression and delivering it to the hearts of many people,” he said. “By the time we mark the 80th or 85th anniversary, we need to come up with a new way of expressing (wartime memories).”

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and original black and white photo that two people walk on a cleared path through the destruction resulting from the Aug. 6 detonation of the first atomic bomb, Sept. 8, 1945. Professor Hidenori Watanabe and his student Anju Niwata of a Tokyo University lab is using AI to add color to historic wartime photographs using a combination of methods. These include the latest AI technologies, but also traditional methods, going door to door interviewing survivors who track back memories to color their family photographs. The team has brought to life hundreds of black-and-white wartime photographs and those of post-war devastation. (U.S. Air ForceAnju Niwata & Hidenori Watanave via AP)

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and its original black and white AP photo that a battered religious figure stands witness on a hill above a burn-razed valley at Nagasaki, on Sept. 24, 1945, after the second atomic bomb ever used in warfare was dropped by the U.S. over the Japanese industrial center. Niwata and Watanave started the photo colorization project in 2018. They call it “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the Pacific War. (U.S. MarinesAnju Niwata & Hidenori Watanave via AP)

This photo combination shows digital colorization, top, by Anju Niwata and Hidenori Watanave, and original black and white photo that a Marine rifleman views the result of the American bombardment of Naha, on Okinawa, June 13, 1945. Niwata and Watanave started the photo colorization project in 2018. They call it “Rebooting Memories,” and they published a book last month of the colorized versions of about 350 monochrome pictures taken before, during and after the Pacific War. (U.S. MarinesAnju Niwata & Hidenori Watanave via AP)

This photo combination shows digital colorization process by Anju Niwata and Hidenori Watanave, left, and original black and white image that Hiroshima resident Hisashi Takahashi and his parents, grandmother and younger brother pose for a photograph in a flower bed of dandelions. The photo was taken around 1935. The original black and white photo is at top right. The second photo, bottom right, shows the colors after AI automatic colorization. The authors initially determined that the flowers in the foreground were white clovers, and adjusted colors of the flower bed accordingly in the third photo, bottom left. After interviewing the photo owner Takahashi, the flowers were found to be dandelions instead of white clovers, and was recolored yellow at top left. Niwata and Watanave call the process “rebooting memories,” whereby colorized photographs revives memories that may otherwise be lost. (Hisashi TakahashiAnju Niwata & Hidenori Watanave via AP)

This photo combination shows digital colorzation, top, by Anju Niwata and Hidenori Watanave, and original photo that Hiroshima resident Hisashi Takahashi, covering his face with watermelon, his family and relatives, pose for a photograph eating watermelon. The photo was taken around 1935. Niwata and Watanave call the process “rebooting memories,” whereby colorized photographs revives memories that may otherwise be lost. (Hisashi TakahashiAnju Niwata & Hidenori Watanave via AP)