Cloud Hosting

Documents have spent decades stubbornly guarding their contents against software. In the late 1960s, the first OCR (optical character recognition) techniques turned scanned documents into raw text. By indexing and searching the text from these digitized documents, software sped up formerly laborious legal discovery and research projects.

Today, Google, Microsoft, and Amazon provide high-quality OCR as part of their cloud services offerings. But documents remain underused in software toolchains, and valuable data languish in trillions of PDFs. The challenge has shifted from identifying text in documents to turning them into structured data suitable for direct consumption by software-based workflows or direct storage into a system of record.

The prevailing assumption is that machine learning, often embellished as AI, is the best way to achieve this, superseding outdated and brittle template-based techniques. This assumption is misguided. The best way to turn the vast majority of documents into structured data is to use the next generation of powerful, flexible templates that find data in a document much as a person would.

The promise of machine learning is that you can train a model once on a large corpus of representative documents and then smoothly generalize to out-of-sample document layouts without retraining. For example, you want to train an ML model on company A, B, and Cs home insurance policies, and then extract the same data from similar documents issued by company Z. This is very difficult to achieve in practice for three reasons:

Your goal is often to extract dozens or hundreds of individual data elements from each document. A model at the document level of granularity will frequently miss some of these values, and those errors are quite difficult to detect. Once your model attempts to extract those dozens or hundreds of data elements from out-of-sample document types, you get an explosion of opportunities for generalization failure.

While some simple documents might have a flat key/value ontology, most will have a substructure:think of a list of deficiencies in a home inspection report or the set of transactions in a bank statement. In some cases youll even encounter complex nested substructures:think of a list of insurance policies, each with a claims history. You either need your machine learning model to infer these hierarchies, or you need to manually parameterize the model with these hierarchies and the overall desired ontology before training.

A document is anything that fits on one or more sheets of paper and contains data! Documents are really just bags of diverse and arbitrary data representations. Tables, labels, free text, sections, images, headers and footers: you name it and a document can use it to encode data. There's no guarantee that two documents, even with the same semantics, will use the same representational tools.

It's no surprise that ML-based document parsing projects can take months, require tons of data up front, lead to unimpressive results, and in general be "grueling" (to directly quote a participant in one such project with a leading vendor in the space).

These issues strongly suggest that the appropriate angle of attack for structuring documents is at the data element level rather than the whole-document level. In other words, we need to extract data from tables, labels, and free text; not from a holistic document. And at the data element level, we need powerful tools to express the relationship between the universe of representational modes found in documents and the data structures useful to software.

So let's get back to templates.

Historically, templates have had an impoverished means of expressing that mapping between representational mode and data structure. For example, they might instruct: go to page 3 and return any text within these box coordinates. This breaks down immediately for any number of reasons, including if:

None of these minor changes to the document layout would faze a human reader.

For software to successfully structure complex documents, you want a solution that sidesteps the battle of months-long ML projects versus brittle templates. Instead, lets build a document-specific query language that (when appropriate) embeds ML at the data element, rather than document, level.

First, you want primitives (i.e., instructions) in the language that describe representational modes (like a label/value pair or repeating subsections) and stay resilient to typical layout variations. For example, if you say:

Find a row starting with this word and grab the lowest dollar amount from it

You want row recognition thats resilient to whitespace variation, vertical jitter, cover pages, and document skew, and you want powerful type detection and filtering.

Second, for data representations with a visual or natural language component, such as tables, checkboxes, and paragraphs of free text, the primitives should embed ML. At this level of analysis, Google, Amazon, Microsoft, and OpenAI all have tools that work quite well off the shelf.

Sensible takes just that approach: blending powerful and flexible templates with machine learning. With SenseML, our JSON-based query language for documents, you can extract structured data from most document layouts in minutes with just a single reference sample. No need for thousands of training documents and months spent tweaking algorithms, and no need to write hundreds of rules to account for tiny layout differences.

SenseMLs wide range of primitives allows you to quickly map representational modes to useful data structures, including complex nested substructures. In cases where the primitives do not use ML, they behave deterministically to provide strong behavior and accuracy guarantees. And even for the non-deterministic output of our ML-powered primitives, such as tables, validation rules can identify errors in the ML output.

What this means is that document parsing with Sensible is incredibly fast, transparent, and flexible. If you want to add a field to a template or fix an error, it's straightforward to do so.

The tradeoff for Sensibles rapid time to value is that each meaningfully distinct document layout requires a separate template. But this tradeoff turns out to be not so bad in the real world. In most business use cases, there are a countable number of layouts (e.g., dozens of trucking carriers generating rate confirmations in the USA; a handful of software systems generating home inspection reports). Our customers dont create thousands of document templates most generate tremendous value with just a few.

Of course, for every widely used tax form, insurance policy, and verification of employment, collectively we only need to create a template once. Thats why weve introduced

Our open-source Sensible Configuration Library is a collection of over 100 of the most frequently parsed document layouts, from auto insurance policies to ACORD forms, loss runs, tax forms, and more. If you have a document that's of broad interest, we'll do the onboarding for you and then make it freely available to the public. It will also be free for you to use for up to 150 extractions per month on our free account tier.

We believe that this hybrid approach is the path to transparently and efficiently solving the problem of turning documents into structured data for a wide range of industries, including logistics, financial services, insurance, and healthcare. If you'd like to join us on this journey and connect your documents to software,schedule a demo or sign up for a free account!

L O A D I N G. . . comments & more!

Link:
Machine Learning is the Wrong Way to Extract Data From Most Documents - hackernoon.com

According to the latest research by SkyQuest Technology, the Global Machine Learning Market was valued at US$ 16.2 billion in 2021, and it is expected to reach a market size of US$ 164.05 billion by 2028, at a CAGR of 39.2 % over the forecast period 20222028. The research provides up-to-date Machine Learning Market analysis of the current market landscape, latest trends, drivers, and overall market environment.

Software systems may forecast events more correctly with the use of machine learning (ML), a type of artificial intelligence (AI), without needing to be explicitly told to do so. Machine learning algorithms use historical data as input to anticipate new output values. As organizations adopt more advanced security frameworks, the global machine learning market is anticipated to grow as machine learning becomes a prominent trend in security analytics. Due to the massive amount of data being generated and communicated over several networks, cyber professionals struggle considerably to identify and assess potential cyber threats and assaults.

Machine-learning algorithms can assist businesses and security teams in anticipating, detecting, and recognising cyber-attacks more quickly as these risks become more widespread and sophisticated. For example, supply chain attacks increased by 42% in the first quarter of 2021 in the US, affecting up to 7,000,000 people. For instance, AT&T and IBM claim that the promise of edge computing and 5G wireless networking for the digital revolution will be proven. They have created virtual worlds that, when paired with IBM hybrid cloud and AI technologies, allow business clients to truly experience the possibilities of an AT&T connection.

Computer vision is a cutting-edge technique that combines machine learning and deep learning for medical imaging diagnosis. This has been accepted by the Microsoft InnerEye programme, which focuses on image diagnostic tools for image analysis. For instance, using minute samples of linguistic data, an AI model created by a team of researchers from IBM and Pfizer can accurately forecast the eventual onset of Alzheimers disease in healthy persons by 71 percent (obtained via clinical verbal cognition tests).

Read Market Research Report, Global Machine Learning Market by Component, (Solutions, and Services), Enterprise Size (SMEs And Large Enterprises), Deployment (Cloud, On-Premise), End-User [Healthcare, Retail, IT and Telecommunications, Banking, Financial Services and Insurance (BFSI), Automotive & Transportation, Advertising & Media, Manufacturing, Others (Energy & Utilities, Etc.)], and Region Forecast and Analysis 20222028 By Skyquest

Get Sample PDF : https://skyquestt.com/sample-request/machine-learning-market

Large enterprises segment dominated the machine learning market in 2021. This is because data science and artificial intelligence technologies are being used more often to incorporate quantitative insights into business operations. For instance, under a contract between Pitney Bowes and IBM, IBM will offer managed infrastructure, IT automation, and machine learning services to help Pitney Bowes convert and adopt hybrid cloud computing to support its global business strategy and goals.

Small and midsized firms are expected to grow considerably throughout the anticipated timeframe. It is projected that AI and ML would be the main technologies allowing SMEs to reduce ICT investments and access digital resources. For instance, the IPwe Platform, IPwe Registry, and Global Patent Marketplace are just a few of the small- and medium-sized firms (SMEs) and other organizations that are reportedly already using IPwes technology.

The healthcare sector had the biggest share the global machine learning market in 2021 owing to the industrys leading market players doing rapid research and development, as well as the partnerships formed in an effort to increase their market share. For instance, per the terms of the two businesses signed definitive agreement, Francisco Partners would buy IBMs healthcare data and analytics assets that are presently a part of the Watson Health company. An established worldwide investment company with a focus on working with IT startups is called Francisco Partners. Francisco Partners acquired a wide range of assets, including Health Insights, MarketScan, Clinical Development, Social Program Management, Micromedex, and imaging software services.

The prominent market players are constantly adopting various innovation and growth strategies to capture more market share. The key market players are IBM Corporation, SAP SE, Oracle Corporation, Hewlett Packard Enterprise Company, Microsoft Corporation, Amazon Inc., Intel Corporation, Fair Isaac Corporation, SAS Institute Inc., BigML, Inc., among others.

The report published by SkyQuest Technology Consulting provides in-depth qualitative insights, historical data, and verifiable projections about Machine Learning Market Revenue. The projections featured in the report have been derived using proven research methodologies and assumptions.

Speak With Our Analyst : https://skyquestt.com/speak-with-analyst/machine-learning-market

Report Findings

What does this Report Deliver?

SkyQuest has Segmented the Global Machine Learning Market based on Component, Enterprise Size, Deployment, End-User, and Region:

Read Full Report : https://skyquestt.com/report/machine-learning-market

Key Players in the Global Machine Learning Market

About Us-SkyQuest Technology Group is a Global Market Intelligence, Innovation Management & Commercialization organization that connects innovation to new markets, networks & collaborators for achieving Sustainable Development Goals.

Find Insightful Blogs/Case Studies on Our Website:Market Research Case Studies

Read more:
Global Machine Learning Market is Expected to Grow at a CAGR of 39.2 % by 2028 - Digital Journal

image:The core of EnCORE: co-principal investigators include (from l to r) Yusu Wang, Barna Saha (the principal investigator), Kamalika Chaudhuri, (top row) Arya Mazumdar and Sanjoy Dasgupta. view more

Credit: University of California San Diego

A new National Science Foundation initiative has created a $10 million dollar institute led by computer and data scientists at University of California San DIego that aims to transform the core fundamentals of the rapidly emerging field of Data Science.

Called The Institute for Emerging CORE Methods in Data Science (EnCORE), the institute will be housed in the Department of Computer Science and Engineering (CSE), in collaboration with The Halcolu Data Science Institute (HDSI), and will tackle a set of important problems in theoretical foundations of Data Science.

UC San Diego team members will work with researchers from three partnering institutions University of Pennsylvania, University of Texas at Austin and University of California, Los Angeles to transform four core aspects of data science: complexity of data, optimization, responsible computing, and education and engagement.

EnCORE will join three other NSF-funded institutes in the country dedicated to the exploration of data science through the NSFs Transdisciplinary Research in Principles of Data Science Phase II (TRIPODS) program.

The NSF TRIPODS Institutes will bring advances in data science theory that improve health care, manufacturing, and many other applications and industries that use data for decision-making, said NSF Division Director for Electrical, Communications and Cyber Systems Shekhar Bhansali.

UC San Diego Chancellor Pradeep K. Khosla said UC San Diegos highly collaborative, multidisciplinary community is the perfect environment to launch and develop EnCORE. We have a long history of successful cross-disciplinary collaboration on and off campus, with renowned research institutions across the nation. UC San Diego is also home to the San Diego Supercomputer Center, the HDSI, and leading researchers in artificial intelligence and machine learning, Khosla said. We have the capacity to house and analyze a wide variety of massive and complex data sets by some of the most brilliant minds of our time, and then share that knowledge with the world.

Barna Saha, the EnCORE project lead and an associate professor in UC San Diegos Department of Computer Science and Engineering and HDSI, said: We envision EnCORE will become a hub of theoretical research in computing and Data Science in Southern California. This kind of national institute was lacking in this region, which has a lot of talent. This will fill a much-needed gap.

The other UC San Diego faculty members in the institute include professors Kamalika Chaudhury, and Sanjoy Dasgupta from CSE; Arya Mazumdar (EnCORE co-principal investigator), Gal Mishne, and Yusu Wang from HDSI; and Fan Chung Graham from Mathematics. Saura Naderi of HDSI will spearhead the outreach activities of the institute.

Professor Barna Saha has assembled a team of exceptional scholars across UC San Diego and across the nation to explore the underpinnings of data science. This kind of institute, focused on groundbreaking research, innovative education and effective outreach, will be a model of interdisciplinary initiatives for years to come, said Department of Computer Science and Engineering Chair Sorin Lerner.

CORE Pillars of Data Science

The EnCORE Institute seeks to investigate and transform three research aspects of Data Science:

EnCORE represents exactly the kind of talent convergence that is necessary to address the emerging societal need for responsible use of data. As a campus hub for data science, HDSI is proud of a compelling talent pool to work together in advancing the field, said HDSI founding director Rajesh K. Gupta.

Team members expressed excitement about the opportunity of interdisciplinary research that the institute will provide. They will work together to improve privacy-preserving machine learning and robust learning, and to integrate geometric and topological ideas with algorithms and machine learning methodologies to tame the complexity in modern data. They envision a new era in optimization with the presence of strong statistical and computational components adding new challenges.

One of the exciting research thrusts at EnCORE is data science for accelerating scientific discoveries in domain sciences, said Gal Mishne, a professor at HDSI. As part of EnCORE, the team will be developing fast, robust low-distortion visualization tools for real-world data in collaboration with domain experts. In addition, the team will be developing geometric data analysis tools for neuroscience, a field which is undergoing an explosion of data at multiple scales.

From K-12 and Beyond

A distinctive aspect to EnCORE will be the E, education and engagement, component.

The institute will engage students at all levels, from K-12 to postdoctoral students, and junior faculty and conduct extensive outreach activities at all of its four sites.

The geographic span of the institute in three regions of the United States will be a benefit as the institute executes its outreach plan, which includes regular workshops, events, hiring of students and postdoctoral students. Online and joint courses between the partner institutions will also be offered.

Activities to reach out to high school, middle school and elementary students in Southern California are also part of the institutes plan, with the first engagement planned for this summer with the Sweetwater Union High School District to teach students about the foundations of data science.

There will also be mentorship and training opportunities with researchers affiliated with EnCORE, helping to create a pipeline of data scientists and broadening the reach and impact of the field. Additionally, collaboration with industry is being planned.

Mazumdar, an associate professor in the HDSI and an affiliated faculty member in CSE, said the team has already put much thought and effort into developing data science curricula across all levels. We aim to create a generation of experts while being mindful of the needs of society and recognizing the demands of industry, he said.

We have made connections with numerous industry partners, including prominent data science techs and also with local Southern California industries including start-ups, who will be actively engaged with the institute and keep us informed about their needs, Mazumdar added.

An interdisciplinary, diverse field- and team

Data science has footprints in computer science, mathematics, statistics and engineering. In that spirit, the researchers from the four participating institutions who comprise the core team have diverse and varied backgrounds from four disciplines.

Data science is a new, and a very interdisciplinary area. To make significant progress in Data Science you need expertise from these diverse disciplines. And its very hard to find experts in all these areas under one department, said Saha. To make progress in Data Science, you need collaborations from across the disciplines and a range of expertise. I think this institute will provide this opportunity.

And the institute will further diversity in science, as EnCORE is being spearheaded by women who are leaders in their fields.

Continued here:
New $10M NSF-funded institute will get to the CORE of data science - EurekAlert