Share this

You might be free to share this text below the Attribution Worldwide license.

Shimon, the marimba-playing robotic, has realized some new abilities, researchers report.

Shimon sings, dances just a little, writes lyrics, and might even compose some melodies.

Now he’s taking them on the street in a live performance tour to help a brand new album—similar to another musician.

“That is the primary time that I truly wrote a tune, as a result of I had inspiration: I had Shimon writing lyrics for me.”

The brand new album may have eight to 10 songs Shimon wrote along with his creator, Gil Weinberg, a professor within the College of Music and the founding director of the Middle for Music Know-how at Georgia Institute of Know-how, the place he leads the Robotic Musicianship group.

Shimon’s album will drop on Spotify later this spring.

Shimon’s reinvention

“Shimon has been reborn as a singer-songwriter,” Weinberg says. “Now we collaborate between people and robots to make songs collectively.”

Take a look at the video for Shimon’s first single, “Into Your Thoughts” right here:

Weinberg will begin with a theme—say, area—and Shimon will write lyrics across the theme. Weinberg places them collectively and composes melodies to suit them. Shimon may also generate some melodies for Weinberg to make use of as he places collectively a tune. Then, with a band of human musicians, Shimon will play the songs and sing.

“I all the time wished to put in writing songs, however I simply can’t write lyrics. I’m a jazz participant,” Weinberg says. “That is the primary time that I truly wrote a tune, as a result of I had inspiration: I had Shimon writing lyrics for me.”

Also Read |  ‘Noisy’ neurons put a restrict on visible notion

Weinberg and his college students have skilled Shimon on datasets of 50,000 lyrics from jazz, prog rock, and hip-hop. Then Shimon makes use of deep studying, a category of machine studying algorithms, to generate his personal phrases.

“There are many methods that use deep studying, however lyrics are totally different,” says Richard Savery, a third-year PhD scholar who has been working with Shimon over the previous 12 months on his songwriting.

“The best way semantic that means strikes via lyrics is totally different. Additionally, rhyme and rhythm are clearly tremendous essential for lyrics, however that isn’t as current in different textual content turbines. So, we use deep studying to generate lyrics, however it’s additionally mixed with semantic information.”

Shimon sits behind marimbas with 5 other musicians on keyboard, drums, and guitars standing around Musical robotic Shimon performs with human musicians on considered one of his new songs. Shimon has realized to put in writing lyrics and sing alongside his human collaborators. (Credit score: Gil Weinberg/Georgia Tech)

Savery provides this instance of the way it may work: “You’ll get a phrase like ‘storm,’ after which it’ll generate a complete bunch of associated phrases, like ‘rain.’ It creates a loop of producing numerous materials, deciding what’s good, after which producing extra based mostly on that.”

When Shimon sings these songs, he actually does sing, with a novel voice created by collaborators at Pompeu Fabra College in Barcelona. They used machine studying to develop the voice and skilled it on a whole bunch of songs.

New sound, new look

Alongside along with his new abilities—all developed in Weinberg’s lab—Shimon has some new , too, that modifications how he performs and strikes on stage. To be clear, he’s nonetheless largely stationary, however he has a mouth, new eyebrows, and new head actions designed to assist convey emotion and work together along with his bandmates. He additionally has new “fingers,” which have completely modified how he performs the marimba.

Also Read |  The Way forward for Aviation Will Be Greener and Quicker Than Ever Earlier than

“We additionally wish to ensure he’s interacting with the opposite musicians round him to present that really feel that he’s performing with individuals.”

“Shimon performs a lot sooner—about 25 to 30 hertz on the most—and in addition far more expressively, enjoying from a gentle dynamic vary to a powerful dynamic vary,” says PhD scholar Ning Yang, who designed all-new motors and for Shimon. “That additionally permits [Shimon] to do choreography through the music being performed.”

For instance, Shimon can depend in at the start of songs to cue the band, and typically he’ll wave his mallets round in time to the music. New brushless DC motors imply he has a a lot larger vary of movement and management of that movement. Yang completed that by bringing his engineering information and musical background collectively to create human-inspired gestures.

He labored carefully with fellow PhD scholar Lisa Zahray, who created a brand new suite of gestures for the robotic—together with how he makes use of these new eyebrows.

“Now we have to consider his function at every time through the tune and what he ought to be doing,” Zahray says. “We additionally wish to ensure he’s interacting with the opposite musicians round him to present that really feel that he’s performing with individuals.”

That partnership with individuals is essential for Weinberg. Educating Shimon new abilities isn’t about changing musicians, he says.

“We are going to want musicians, and there might be extra musicians that can be capable to do extra and new music as a result of robots will assist them, will generate concepts, will assist them broaden the way in which they give thought to music and play music,” Weinberg says.

Also Read |  This Startup Desires to Retailer Grid Power by Lifting Concrete Blocks

Shimon, Weinberg, and your complete band are constructing a touring schedule now with the purpose of taking their distinctive mix of robot- and human-created music to extra individuals. Weinberg says he hopes these exhibits will show to be greater than a novelty act.

“I believe we’ve got reached a degree the place I anticipate the viewers to only benefit from the music for music’s sake,” Weinberg says.

“That is music that people, by themselves, wouldn’t have written. I would like the viewers to suppose, ‘There’s one thing distinctive about this tune, and I wish to return and hearken to it, even when I don’t take a look at the robotic.’”

Supply: Georgia Tech