icon bookmark-bicon bookmarkicon cameraicon checkicon chevron downicon chevron lefticon chevron righticon chevron upicon closeicon v-compressicon downloadicon editicon v-expandicon fbicon fileicon filtericon flag ruicon full chevron downicon full chevron lefticon full chevron righticon full chevron upicon gpicon insicon mailicon moveicon-musicicon mutedicon nomutedicon okicon v-pauseicon v-playicon searchicon shareicon sign inicon sign upicon stepbackicon stepforicon swipe downicon tagicon tagsicon tgicon trashicon twicon vkicon yticon wticon fm

AI proves ‘too good’ at writing fake news, held back by researchers

AI proves ‘too good’ at writing fake news, held back by researchers
OpenAI, the non-profit founded by Elon Musk and Sam Altman, is withholding its newly-developed AI writer for fear that it could be weaponized by unscrupulous actors to mass produce convincing fake news.

The organization created a machine learning algorithm, GPT-2, that can produce natural-looking language largely indistinguishable from that of a human writer while largely “unsupervised” – it needs only a small prompt text to provide the subject and context for the task.  

The team have made some strides toward this lofty goal, but have also somewhat inadvertently admitted that, once perfected, the device can mass-produce fake news on an unprecedented scale. A fake news super weapon for the information warfare era, if you will.

“We have observed various failure modes,” the team observed. “Such as repetitive text, world modelling failures (eg the model sometimes writes about fires happening under water), and unnatural topic switching.”

With topics familiar to the system (a large online footprint with plenty of sources e.g. news about Ariana Grande, Hillary Clinton etc) the system can generate “reasonable sample” roughly 50 percent of the time.

“Overall, we find that it takes a few tries to get a good sample” says David Luan, vice president of engineering at OpenAI.

Also on rt.com Pentagon’s 1st AI strategy vows to keep pace with Russia & China, wants help from tech

GPT-2 boasts 1.5 billion parameters, and was trained on a far larger dataset than its next nearest competitors and the system employs machine learning to establish “quality” sources of content, based on some eight million pages posted to link-sharing site Reddit. For a link to qualify for inclusion, it needs a “karma” score of three or higher, meaning that three human minds deemed the link worthy of viewing.

“This can be thought of as a heuristic indicator for whether other users found the link interesting, educational or just funny,” the team writes.

Also on rt.com AI can’t beat a human in a debate (yet), but give it time…

Quotes and attributions are entirely fabricated by GPT-2, but the story, constructed word-by-word, is coherent and based entirely on pre-existing content online while avoid direct plagiarism. Critics have already highlighted that the paper published alongside OpenAI's announcement has not been peer-reviewed.

Debate is already raging online about the moral and ethical implications of such technology and its potential impact on the online information ecosystem as well as the political process around the wider, physical world.

“[We] think governments should consider expanding or commencing initiatives to more systematically monitor the societal impact and diffusion of AI technologies, and to measure the progression in the capabilities of such systems,” OpenAI said.

Google parent company Alphabet has adopted a similar practice of not divulging its latest AI research openly to the public for fear that it may be weaponized.

Think your friends would be interested? Share this story!