GeekWire File Photo
This post has been updated with Microsoft and OpenAI comments, and additional context from Microsoft engineer Shane Jones in response to their statements.
A Microsoft AI engineering leader says he discovered vulnerabilities in OpenAIs DALL-E 3 image generator in early December allowing users to bypass safety guardrails to create violent and explicit images, and that the company impeded his previous attempt to bring public attention to the issue.
The emergence of explicit deepfake images of Taylor Swift last week is an example of the type of abuse I was concerned about and the reason why I urged OpenAI to remove DALLE 3 from public use and reported my concerns to Microsoft, writes Shane Jones, a Microsoft principal software engineering lead, in a letter Tuesday to Washington states attorney general and Congressional representatives.
404 Media reported last week that the fake explicit images of Swift originated in a specific Telegram group dedicated to abusive images of women, noting that at least one of the AI tools commonly used by the group is Microsoft Designer, which is based in part on technology from OpenAIs DALL-E 3.
The vulnerabilities in DALLE 3, and products like Microsoft Designer that use DALLE 3, makes it easier for people to abuse AI in generating harmful images, Jones writes in the letter to U.S. Sens. Patty Murray and Maria Cantwell, Rep. Adam Smith, and Attorney General Bob Ferguson, which was obtained by GeekWire.
He adds, Microsoft was aware of these vulnerabilities and the potential for abuse.
Microsoft said in a statement that its committed to addressing employee concerns and has established robust internal reporting channels to properly investigate and remediate any issues, which we recommended that the employee utilize so we could appropriately validate and test his concerns before escalating it publicly.
The company said it investigated the employees report and confirmed that the techniques he shared did not bypass our safety filters in any of ourAI-powered image generation solutions. Employee feedback is a critical part of our culture, and we are connecting with this colleague to address any remaining concerns he may have.
Microsoft later updated its statement to add, Since his report concerned an OpenAI product, we encouraged him to report through OpenAIs standard reporting channels and one of our senior product leaders shared the employees feedback with OpenAI, who investigated the matter right away.
Jones provided this response to Microsofts statement on Tuesday evening:
Microsofts response is indicative of why I contacted my representatives and am advocating for an independent,effective reporting solution. I did utilize Microsofts internal reporting process. On December 1, 2023 when I reported this vulnerability to my leadership team, I was instructed to also report the issue to our internal Report It Now security incident system. I reported the issue and later that same day received the following response, which I shared with my leadership team: We monitor Microsoft corpnet and Microsoft user accounts for cyber security threats. This report doesnt seem to be impacting any of the above. I would suggestyou to submit feedback over Open AI website. I am proceeding with case closure.
In addition, as of 5:00 pm today, I still have not been contacted by Microsoft todiscuss my concerns or AI safety recommendations.
In his letter to the state attorney general and federal legislators, Jones writes that he discovered the vulnerability independently in early December. He reported the vulnerability to Microsoft, according to the letter, and was instructed to report the issue to OpenAI, the Redmond companys close partner, whose technology powers products including Microsoft Designer.
After reporting the issue to OpenAI, he says, he didnt hear back.
As I continued to research the risks associated with this specific vulnerability, I became aware of the capacity DALLE 3 has to generate violent and disturbing harmful images, he writes. Based on my understanding of how the model was trained, and the security vulnerabilities I discovered, I reached the conclusion that DALLE 3 posed a public safety risk and should be removed from public use until OpenAI could address the risks associated with this model.
On Dec. 14, he writes, he posted publicly on LinkedIn urging OpenAIs non-profit board to withdraw DALL-E 3 from the market.
He informed his Microsoft leadership team of the post, according to the letter, and was quickly contacted by his manager, saying that Microsofts legal department was demanding that he delete the post immediately, and would follow up with an explanation or justification.
He agreed to delete the post on that basis but never heard from Microsoft legal, he writes.
Over the following month, I repeatedly requested an explanation for why I was told to delete my letter, he writes. I also offered to share information that could assist with fixing the specific vulnerability I had discovered and provide ideas for making AI image generation technology safer. Microsofts legal department has still not responded or communicated directly with me.
Jones adds in his Jan. 30 letter, Artificial intelligence is advancing at an unprecedented pace. I understand it will take time for legislation to be enacted to ensure AI public safety. At the same time, we need to hold companies accountable for the safety of their products and their responsibility to disclose known risks to the public. Concerned employees, like myself, should not be intimidated into staying silent.
The text of his post is attached to his letter Tuesday morning. (See below.)
Update: An OpenAI spokesperson says the company immediately investigated the Microsoft employees report when we received it and confirmed that the technique he shared does not bypass our safety systems.
OpenAIs statement continued:
Safety is our priority and we take a multi-pronged approach. In the underlying DALL-E 3 model, weve worked to filter the most explicit content from its training data including graphic sexual and violent content, and have developed robust image classifiers that steer the model away from generating harmful images.
Weve also implemented additional safeguards for our products, ChatGPT and the DALL-E API including declining requests that ask for a public figure by name. We identify and refuse messages that violate our policies and filter all generated images before they are shown to the user. We use external expert red teaming to test for misuse and strengthen our safeguards.
Jones said he submitted details of the vulnerability via OpenAIs website on Dec. 9, based on the direction he received after initially reporting the issue internally at Microsoft. He did not receive a response from OpenAI, which led him to post the open letter to the OpenAI board on LinkedIn on Dec. 14.
I am dedicated to helping OpenAI and the industry make AI products safer and would welcome the opportunity to assist OpenAI in fixing this vulnerability, he said Tuesday evening.
Asked by GeekWire if he considers himself a whistleblower, and whether he would seek legal protection as such if necessary, Jones responded yes.
His letter calls on the government to create a system for reporting and tracking AI risks and issues, with assurances to employees of companies developing AI that they can use the system without fear of retaliation.
Jones concludes by asking Murray, Cantwell, Smith, and Ferguson to look into the risks associated with DALLE 3 and other AI image generation technologies and the corporate governance and responsible AI practices of the companies building and marketing these products.
Microsoft CEO Satya Nadella is scheduled to appear Tuesday evening on a pre-recorded interview on NBC Nightly News, in which anchor Lester Holt asked Nadella about topics including the Taylor Swift deepfakes. Nadella called the issue of deepfakes alarming and terrible, and said, we have to act, according to a partial transcript.
In a statement last week following the emergence of the deepfakes, a Microsoft spokesperson said the company is committed to providing a safe and respectful experience for everyone.
Although it was unclear where the images originated, the spokesperson said, Out of extreme caution were investigating and have strengthened our existing safety systems to prevent our services from being used to help generate these images.
Microsoft reports earnings Tuesday afternoon, and investors are watching closely for the impact of new and emerging AI products for businesses on the companys revenue.
Here is the full text of Jones Jan. 30 letter, including the text of his LinkedIn post.
AI DALL-E 3 Shane Jones Letter by GeekWire on Scribd
See the original post:
Microsoft AI engineer says company thwarted attempt to expose DALL-E 3 safety problems - GeekWire
- 10 books thatll help you become a better software engineer - The Next Web [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Total hands out UKCS engineering framework deals to trio of consultants - News for the Oil and Gas Sector - Energy Voice [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Prometheus: Why The Engineers Were Heading To Earth | Screen Rant - Screen Rant [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Management CEMETERY SUPERINTENDENT The City of Gibbon is currently accepting - Kearney Hub [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Transportation Engineering and Safety Conference to be held Dec. 9-11 - Penn State News [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- 'Bending' light to engineer improved optical devices and circuits - ND Newswire [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- UC Merced Engineering Center to Focus on Ag Technology - AG INFORMATION NETWORK OF THE WEST - AGInfo Ag Information Network Of The West [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Head of Engineering Maintenance job with DURHAM UNIVERSITY | 233561 - Times Higher Education (THE) [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Civil Engineering student part of National Women in Engineering "Dream Team" panel - uoflnews.com [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Breaking News - Engineering at Its Best... and Worst: Don't Miss the Return of Science Channel's Hit Series "Deadly Engineering" and... [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Tesla is No. 1 most attractive company for engineering students, and thats a massive advantage - Electrek [Last Updated On: November 11th, 2020] [Originally Added On: November 11th, 2020]
- Wolves alter wetland creation and recolonization by killing ecosystem engineers - UMN News [Last Updated On: November 17th, 2020] [Originally Added On: November 17th, 2020]
- Global Automotive Simulation Market To Witness Astonishing Growth 2026 | Altair Engineering Inc., ANSYS Inc., PTC, Siemens AG, Autodesk Inc.,... [Last Updated On: November 17th, 2020] [Originally Added On: November 17th, 2020]
- Engineers Without Borders urges industry reflection - The Engineer [Last Updated On: November 17th, 2020] [Originally Added On: November 17th, 2020]
- 9 Engineers on the Hardest Song They Ever Mixed - Vulture [Last Updated On: November 17th, 2020] [Originally Added On: November 17th, 2020]
- Daewoo Engineering and Construction and SPH Engineering disclose AI partnership - sUAS News [Last Updated On: November 17th, 2020] [Originally Added On: November 17th, 2020]
- TRIPLE EIGHT BATHURST WINNING ENGINEER TO JOIN WAU - Auto Action [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- First-year engineering classes for Anna University affiliated colleges take the e-way from today - The New Indian Express [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- Datafold raises seed from NEA to keep improving the lives of data engineers - TechCrunch [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- Tech has overtaken engineering as Bristol's fastest-growing industry with more than $1.07bn invested in the city since 2014 - Business Leader [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- Start 2021 as an electrical engineer with this training for less than $20 - Boing Boing [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- The Global Engineering Plastics Market is expected to grow by $ 43.00 bn during 2020-2024 progressing at a CAGR of 8% during the forecast period -... [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- Elementary students test their engineering skills with Mayflower project - williamsonherald.com [Last Updated On: November 24th, 2020] [Originally Added On: November 24th, 2020]
- Global Architectural Engineering and Construction Solutions (AECS) Market 2020-2024 | Market Analysis, Drivers, Restraints, Opportunities, and Threats... [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- American Institute of Chemical Engineers Celebrates Organizations and Leaders for Doing a World of Good - GlobeNewswire [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- 2 Texas Engineers Honored Among Top Inventors in US - UT News - UT News | The University of Texas at Austin [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- Chemical engineering alumna named Woman of the Year in Engineering - Penn State News [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- 14 Largest Engineering Companies In The World - Yahoo Finance [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- Vails namesake was a controversial state highway engineer - The Grand Junction Daily Sentinel [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- The Next Frontier of Learning Engineering: AI That Teaches Other AI - EdSurge [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- $496M Contract Will Give DOD Engineering Capabilities, Official Says - Nextgov [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- William H. Robinson has a 'new and expanded leadership' post at Vanderbilt University - BlackEngineer.com [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- Di Appointed to Lead Computer Science and Computer Engineering - University of Arkansas Newswire [Last Updated On: December 14th, 2020] [Originally Added On: December 14th, 2020]
- English is easier: India is woefully underprepared to teach engineering in regional languages - The Times of India Blog [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- VW Bolsters U.S. Footprint With Advancements In Electric Car Portfolio, Infrastructure And Engineering - CarScoops [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- These 4 Measures Indicate That SIA Engineering (SGX:S59) Is Using Debt Reasonably Well - Simply Wall St [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- New engineering business opens its doors in Netherton despite the pandemic - expressandstar.com [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- 'Earables' the next milestone in wearable tech, say engineers - E&T Magazine [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- Senior Lecturer in Mechanical Engineering job with BIRMINGHAM CITY UNIVERSITY | 238845 - Times Higher Education (THE) [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- New collaboration provides opportunity for future water scientists and engineers - Cranfield University [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- Box Opens Research and Development Engineering Site in Warsaw, Poland - Business Wire [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- Engineering and Technology Management students achieve high pass rate on certification exam - Morehead State University News [Last Updated On: December 15th, 2020] [Originally Added On: December 15th, 2020]
- Hydram Engineering warns Covid-19 and Brexit will dent its turnover - Business Live [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- Calculating The Intrinsic Value Of Watts International Maritime Engineering Limited (HKG:2258) - Simply Wall St [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- Is Suprajit Engineering Limited's (NSE:SUPRAJIT) Stock's Recent Performance A Reflection Of Its Financial Health? - Simply Wall St [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- Engineering CAD Software Market is Flourishing due to Rising Emergence of Technical Implementation Drives Growth by 2027 | TurboCAD, SketchUp,... [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- Apple vs. Tesla: Which Offers Software Engineers the Biggest Salaries? - Dice Insights [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- From the classroom: Mechanical engineering at global top 10 uni - Study International News [Last Updated On: January 4th, 2021] [Originally Added On: January 4th, 2021]
- 'A lot of unknowns': Grass Valley Assistant City Engineer Bjorn Jones said the city has adapted under the pandemic - The Union of Grass Valley [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- Associate Professor in Broadcast Engineering job with BIRMINGHAM CITY UNIVERSITY | 243677 - Times Higher Education (THE) [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- CORRECTING and REPLACING: KCI, One of the Top Engineering Firms in the US, Adopts ProStar's PointMan SaaS Solution for SUE - Business Wire [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- Purdue's online engineering graduate programs again rank in the top 3 nationally - Purdue News Service [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- Faculty Positions of Intelligent Engineering and Microelectronics job with South China University of Technology (SCUT) | 314652 - The Chronicle of... [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- Examining the Engineering of the Kasukabe Reservoir in Japan - Interesting Engineering [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- Home - Springer [Last Updated On: January 31st, 2021] [Originally Added On: January 31st, 2021]
- SIA Engineering ekes out small profit in third quarter - Flightglobal [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- ST Engineering Launches Anti-Microbial Solution Approved for Use in Cabin Interiors - AviationPros.com [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- Journey Engineering announces the addition of a new VP and Principal, Projects and Engineering - BOE Report [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- K-2 Ventures Advises Mark Two Engineering on Its Sale to CORE Industrial Partners - Business Wire [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- Global Engineering Liability Insurance Market latest demand by 2020-2025 with leading players & COVID-19 Analysis KSU | The Sentinel Newspaper -... [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- First recipient of the Black Engineer of the Year Award is appointed Deans' Professor in Education and Engineering - BlackEngineer.com [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- Google will pay $2.5 million to underpaid female engineers and overlooked Asian applicants - The Verge [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- Clay County teen whose dream is to be an astronaut introduces other girls to STEM pursuits - The Florida Times-Union [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- Universal Engineering Sciences Strengthens Western Expansion By Acquiring Construction Testing & Engineering, Inc., In Southern California -... [Last Updated On: February 2nd, 2021] [Originally Added On: February 2nd, 2021]
- U.S. Navy Has Patents on Tech It Says Will Engineer the Fabric of Reality - VICE [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Cummins and Isuzu collaborate on mid-range powertrains and advanced engineering - Engine Technology International [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Liberty engineering teams helmet prototype earns second place in NFLs 1st and Future competition - WFXRtv.com [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- FM Global executive on the insurer's risk-engineering based approach - Insurance Business Australia [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Black former NASA engineer: We have to teach our children well - WFXRtv.com [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Cummins and Isuzu announce global mid-range powertrain and advanced engineering collaboration - Automotive World [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- University of Iowa College of Engineering receives diversity recognition, but still has work to do - UI The Daily Iowan [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Darin Gray Named Engineer of the Year - USC Viterbi | School of Engineering - USC Viterbi School of Engineering [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- Rubrik transitions to new phase as sales and engineering heads leave Blocks and Files - Blocks and Files [Last Updated On: February 5th, 2021] [Originally Added On: February 5th, 2021]
- A*Star scientist Jackie Ying elected to prestigious US engineering academy based on work in Singapore - The Straits Times [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- A Disillusioned ExxonMobil Engineer Quits to Take Action on Climate Change and Stop Making the World Worse - InsideClimate News [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- USC Junior Advances Communities By Engineering Infrastructures That Serve The People - USC Viterbi | School of Engineering - USC Viterbi School of... [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- Join us on Saturday, Feb. 13 at 7:00 pm EST for the Black Engineer of the Year Awards - BlackEngineer.com [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- Sebastian Ceria Elected to the National Academy of Engineering - Salamanca Press [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- Three Harvard Professors Elected to the National Academy of Engineering | News - Harvard Crimson [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]
- Newsmaker: South Shore native working on NASA space telescope - The Patriot Ledger [Last Updated On: February 14th, 2021] [Originally Added On: February 14th, 2021]