CAMBRIDGE, Mass., Jan. 9, 2024 /PRNewswire/ -- MIT Sloan Management Reviewreveals the insights of more than 500 senior data and technology executives in"Five Key Trends in AI and Data Science for 2024," a part of its AI in Actionseries.

Artificial intelligence and data science became front-page news in 2023 thanks to generative AI, state coauthorsThomas H. Davenport, the President's Distinguished Professor of Information Technology and Management at Babson College and a fellow of the MIT Initiative on the Digital Economy, and Randy Bean, an industry thought leader who currently serves as innovation fellow, data strategy, for global consultancy Wavestone.

Orgs need to continually adjust data science skills and processes to get the full value from data, analytics, and AI.

To find out what might keep it on the front page in 2024, Davenport and Bean, during the past several months, conducted three surveys involving more than 500 executives closest to companies' data science and AI strategies to bring to light what organizations are thinking and doing.

"Data science is increasingly critical to every organization. But it's not a static discipline, and organizations need to continually adjust data science skills and processes to get the full value from data, analytics, and AI," said Davenport.

"Expect 2024 to be a year of transformation and change driven by adoption of AI and a reshaping of the data, analytics, and AI leadership role within leading companies," added Bean. "With 33% of midsize to large organizations having appointed or in search of a chief AI officer, and with 83.2% of leading companies having a chief data and analytics officer in place today, it is inevitable that we will witness consolidation of roles, restructuring of responsibilities, elimination of some positions, and some critical rethinking of data and AI leadership expectations during the course of 2024."

"Five Key Trends in AI and Data Science for 2024"culls the surveys to identify developing issues that should be on every leader's radar screen this year:

Generative AI sparkles but needs to deliver value. Survey responses suggest that although excitement is high, the value of generative AI has not been delivered. Large percentages of respondents believe the technology has the potential to be transformational; 80% in one survey said they believe it will transform their organizations, and 64% in another survey said it is the most transformational technology in a generation. A large majority of survey takers are also increasing investment in the technology.

Data science is shifting from artisanal to industrial. Companies are investing in platforms, processes and methodologies, feature stores, machine learning operations (MLOps) systems, and other tools to increase productivity and deployment rates. Automation is helping to increase productivity and enable broader data science participation.

Two versions of data products will dominate. Eighty percent of data and technology leaders in one survey said that their organizations were using or considering the use of data products and product management. But they mean two different things by "data products." Just under half (48%) of respondents said that they include analytics and AI capabilities in the concept of data products. Some 30% view analytics and AI as separate from data products and presumably reserve that term for reusable data assets alone. What matters is that an organization is consistent in how it defines and discusses data products.

Data scientists will become less sexy. The proliferation of roles such as data engineers that can address pieces of the data science problem along with the rise of citizen data science, where savvy businesspeople create models or algorithms themselves is causing the star power of data scientists to recede.

Data, analytics, and AI leaders are becoming less independent.In 2023, increasing numbers of organizations cut back on the proliferation of technology and data "chiefs," including chief data and analytics officers (and sometimes chief AI officers). The functions performed by data and analytics executives haven't gone away; rather, they're increasingly being subsumed within a broader set of technology, data, and digital transformation functions managed by a "supertech leader" who usually reports to the CEO. In 2024, expect to see more of these overarching tech leaders who have all the capabilities to create value from the data and technology professionals reporting to them.

The MIT Sloan Management Review article "Five Key Trends in AI and Data Science for 2024"publishes at 8 a.m. ET on Jan. 9, 2024. This column is part of the series AI in Action.

About the AuthorsThomas H. Davenportis the President's Distinguished Professor of Information Technology and Management at Babson College, a fellow of the MIT Initiative on the Digital Economy, and senior adviser to the Deloitte Chief Data and Analytics Officer Program.He is coauthor of All-In On AI: How Smart Companies Win Big With Artificial Intelligence(HBR Press, 2023) and Working With AI: Real Stories of Human-Machine Collaboration(MIT Press, 2022). Randy Beanis an industry thought leader, author, founder, and CEO and currently serves as innovation fellow, data strategy, for global consultancy Wavestone. He is the author of Fail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data, and AI(Wiley, 2021).

About MIT Sloan Management ReviewMIT Sloan Management Reviewis an independent, research-based magazine and digital platform for business leaders published at the MIT Sloan School of Management. MIT SMR explores how leadership and management are transforming in a disruptive world. We help thoughtful leaders capture the exciting opportunities and face down the challenges created as technological, societal, and environmental forces reshape how organizations operate, compete, and create value.

Connect with MIT Sloan Management Review on:

Tess Woods[emailprotected] 617-942-0336

SOURCE MIT Sloan Management Review

Visit link:

Five Key Trends in AI and Data Science for 2024 From MIT Sloan Management Review - PR Newswire

Ian Bogost, center, of Washington University in St. Louis speaks during a panel discussion at the AI and Data Science Across Disciplines Symposium on Nov. 30 at the University of Georgia Center for Continuing Education & Hotel. Meg Mittelstadt, left, director of UGAs Center for Teaching and Learning, and Tianming Liu, right, Distinguished Research Professor in the School of Computing, and Youjin Kong (not pictured), assistant professor in the department of philosophy, joined Bogost in discussing advances in artificial intelligence. (Photo by Mike Wooten)

Faculty from across the University of Georgia campus gathered on Nov. 30 to discuss the expanding influence of artificial intelligence, share insights into their research and consider how AI may shape higher education and society in the future.

The universitys inaugural Artificial Intelligence and Data Science Across Disciplines Symposium was hosted by the Institute for Artificial Intelligence with support from the Office of the Senior Vice President for Academic Affairs and Provost, the Office of Research and the Franklin College of Arts of Sciences.

The symposium is part of a series of events geared toward bringing together the AI and data science faculty at UGA, said Khaled M. Rasheed, director of the Institute for Artificial Intelligence and a professor in the School of Computing. It was an exciting opportunity for the AI community at UGA to connect and learn.

The symposium, held at the University of Georgia Center for Continuing Education & Hotel, showcased UGAs significant investments in the fields of artificial intelligence and data science. Those investments include an ambitious presidential interdisciplinary faculty hiring initiative that aims to recruit 70 faculty members with expertise in applying data science and artificial intelligence to some of societys most urgent challenges.

Rather than being housed exclusively in a single department, the majority of UGAs newly recruited faculty will focus on the fusion of data science and AI in cross-cutting areas such as infectious diseases, integrative precision agriculture, ethics, cybersecurity, resilient communities and the environment.

The breadth of experience and expertise at the University of Georgia uniquely positions our institution to advance AI and data science scholarship and research, said Jeanette Taylor, the universitys vice provost for academic affairs. We are able to integrate perspectives from a diverse array of disciplines as we consider not only potential uses for AI but also the ethical and social questions that arise.

Ian Bogost, Barbara and David Thomas Distinguished Professor at Washington University in St. Louis with a dual appointment as professor and director of film and media studies and professor of computer science and engineering, provided the symposiums keynote address.

Bogost urged attendees to avoid viewing generative AI, such as ChatGPT and Dall-E, as a tool for process optimization at the expense of imagination.

AI works best for me when I use it to extend my imagination, he said.

The symposium also featured two lightning rounds of brief talks by UGA faculty members from a wide range of disciplines. Faculty highlighted their use of AI and data science in research topics such as crop modeling and assessment, physics-informed machine learning for infectious disease forecasting, data science in advanced manufacturing and AIs integration into society.

A panel discussion closed the symposium. Participants examined the impact of AI and ChatGPT on teaching and learning at UGA, industries that stand to benefit from AI and the ethics of AI in research and society, among other topics.

Building upon the momentum of the symposium, UGAs Office of Research will host an AI Team Acceleration Event on Feb. 5 at the Delta Innovation Hub. This event will include presentations from research teams funded by Presidential Interdisciplinary Seed Grants and an overview of major university resources available to research teams.

UGA is now gathering input from faculty regarding potential interdisciplinary research collaborations. The Office of Research will filter those responses through AI to identify affinity groups faculty can join, and the AI Team Acceleration Event will include time for those groups to meet and begin discussions of possible research projects.

See original here:

Symposium highlights UGAs interdisciplinary AI and data science research and scholarship - University of Georgia

Data analytics is a rapidly growing field that requires a combination of technical and analytical skills. With so many courses available online, it can be challenging to choose the right one. In 2024, for individuals looking for the best Data Analytics courses in the USA, programs typically cover data interpretation, website tracking codes, marketing campaigns, and program management. Emphasis is often placed on integrating analytics into various business facets. The USA, a premier destination for international students, boasts globally ranked universities and diverse study locations. A masters degree follows the completion of an undergraduate program. In this article, we will explore the best data analytics courses in the USA to enroll in 2024.

University: Canisius University

Campus location: Buffalo, USA

Duration: 1-3 years

Tuition fee: US$910/per credit

Course Description: With the MS in Data Analytics in the USA program at Canisius University, set off on a life-changing adventure. The curriculum lasts one to three years and is based in Buffalo, USA. It offers a thorough education in the ever-evolving subject of data analytics for US$910 per credit.

Enroll now

University: Pacific Lutheran University

Campus location: Tacoma, USA

Duration: 9-21 months

Tuition fee: US$1,104 / per credit

Course Description:

Pacific Lutheran University offers a dynamic Master of Science in Marketing Analytics in Tacoma, USA. This 9-21 month program equips students with strategic insights. The tuition is USD 1,104 per credit, ensuring a high-quality education in the heart of the Pacific Northwest.

Enroll now

University: Drew University

Campus location: Madison, USA

Duration: 1-2 years

Tuition fee: US$ 22,248 / per credit

Course Description: Get a data analytics certification program, Drew University offers a cutting-edge Master of Science in Data Science program in Madison, USA. Designed to be completed in 1-2 years, this advanced degree provides a comprehensive exploration of data science, equipping students with the skills and knowledge needed for success in this dynamic field.

Enroll now

University: Alliant International University

Campus Location: San Diego, USA

Duration: 1 year

Tuition fee: US$ 768 / per credit

Course Description: Join the MS program in Healthcare Analytics at Alliant International University in San Diego, USA, to start a life-changing adventure. Immerse yourself in cutting-edge insights at a lively campus in only one year, opening the door to a fascinating career in the rapidly changing field of healthcare analytics.

Enroll now

University: Illinois Institute of Technology

Campus Location: Chicago, USA

Duration: 2 years

Tuition fee: US$ 1,712 / per credit

Course Description:

With the Illinois Institute of Technologys Master of Science in Sustainability Analytics and Management, set off on a life-changing adventure. This two-year, STEM-designated program in the heart of Chicago gives students access to cutting-edge perspectives. It is a forward-thinking investment in a sustainable future, valued at US$1,712 per credit.

Enroll now

University: Southern Methodist University

Campus Location: Dallas, USA

Duration: 2 years

Tuition fee: US$ 74,000

Course Description:

The MS in Applied Statistics and Data Analytics at Southern Methodist University, located in Dallas, USA, spans two years. This program promises a comprehensive exploration of statistical methodologies and data analytics, preparing students for impactful roles in the dynamic field of data science.

Enroll now

University: Mercyhurst University

Campus Location: Erie, USA

Duration: 2 years

Tuition fee: USD 33,000 / per year

Course Description:

Pursue a Master of Science in Applied Intelligence at Mercyhurst University in the United States and set off on a revolutionary adventure. The two-year curriculum, which is based in Erie, provides a dynamic combination of theoretical knowledge and real-world application in the constantly changing field of applied intelligence.

Enroll now

Join our WhatsApp and Telegram Community to Get Regular Top Tech Updates

Read the rest here:

Best Data Analytics Courses in the USA to Enroll in 2024 - Analytics Insight

Components of whylogs

Lets begin by understanding the important characteristics of whylogs.

This is all we need to know about whylogs. If youre curious to know more, I encourage you to check the documentation. Next, lets work to set things up for the tutorial.

Well use a Jupyter notebook for this tutorial. To make our code work anywhere, well use JupyterLab in Docker. This setup installs all needed libraries and gets the sample data ready. If youre new to Docker and want to learn how to set it up, check out this link.

Start by downloading the sample data (CSV) from here. This data is what well use for profiling and validation. Create a data folder in your project root directory and save the CSV file there. Next, create a Dockerfile in the same root directory.

This Dockerfile is a set of instructions to create a specific environment for the tutorial. Lets break it down:

By now your project directory should look something like this.

Awesome! Now, lets build a Docker image. To do this, type the following command in your terminal, making sure youre in your projects root folder.

This command creates a Docker image named pyspark-whylogs. You can see it in the Images tab of your Docker Desktop app.

Next step: lets run this image to start JupyterLab. Type another command in your terminal.

This command launches a container from the pyspark-whylogs image. It makes sure you can access JupyterLab through port 8888 on your computer.

After running this command, youll see a URL in the logs that looks like this: http://127.0.0.1:8888/lab?token=your_token. Click on it to open the JupyterLab web interface.

Great! Everythings set up for using whylogs. Now, lets get to know the dataset well be working with.

Well use a dataset about hospital patients. The file, named patient_data.csv, includes 100k rows with these columns:

As for where this dataset came from, dont worry. It was created by ChatGPT. Next, lets start writing some code.

First, open a new notebook in JupyterLab. Remember to save it before you start working.

Well begin by importing the needed libraries.

Then, well set up a SparkSession. This lets us run PySpark code.

After that, well make a Spark dataframe by reading the CSV file. Well also check out its schema.

Next, lets peek at the data. Well view the first row in the dataframe.

Now that weve seen the data, its time to start data profiling with whylogs.

To profile our data, we will use two functions. First, theres collect_column_profile_views. This function collects detailed profiles for each column in the dataframe. These profiles give us stats like counts, distributions, and more, depending on how we set up whylogs.

Each column in the dataset gets its own ColumnProfileView object in a dictionary. We can examine various metrics for each column, like their mean values.

whylogs will look at every data point and statistically decide wether or not that data point is relevant to the final calculation

For example, lets look at the average height.

Next, well also calculate the mean directly from the dataframe for comparison.

But, profiling columns one by one isnt always enough. So, we use another function, collect_dataset_profile_view. This function profiles the whole dataset, not just single columns. We can combine it with Pandas to analyze all the metrics from the profile.

We can also save this profile as a CSV file for later use.

The folder /home/jovyan in our Docker container is from Jupyter's Docker Stacks (ready-to-use Docker images containing Jupyter applications). In these Docker setups, 'jovyan' is the default user for running Jupyter. The /home/jovyan folder is where Jupyter notebooks usually start and where you should put files to access them in Jupyter.

And thats how we profile data with whylogs. Next, well explore data validation.

For our data validation, well perform these checks:

Now, lets start. Data validation in whylogs starts from data profiling. We can use the collect_dataset_profile_view function to create a profile, like we saw before.

However, this function usually makes a profile with standard metrics like average and count. But what if we need to check individual values in a column as opposed to the other constraints, that can be checked against aggregate metrics? Thats where condition count metrics come in. Its like adding a custom metric to our profile.

Lets create one for the visit_date column to validate each row.

visit_date_condition = {"is_date_format": Condition(Predicate().is_(check_date_format))}

Once we have our condition, we add it to the profile. We use a Standard Schema and add our custom check.

Then we re-create the profile with both standard metrics and our new custom metric for the visit_date column.

With our profile ready, we can now set up our validation checks for each column.

constraints = builder.build()constraints.generate_constraints_report()

We can also use whylogs to show a report of these checks.

Itll be an HTML report showing which checks passed or failed.

Heres what we find:

Lets double-check these findings in our dataframe. First, we check the visit_date format with PySpark code.

+----------+-----+|null_check|count|+----------+-----+|not_null |98977||null |1023 |+----------+-----+

It shows that 1023 out of 100,000 rows dont match our date format. Next, the weight column.

+------+-----+|weight|count|+------+-----+|0 |2039 |+------+-----+

Again, our findings match whylogs. Almost 2,000 rows have a weight of zero. And that wraps up our tutorial. You can find the notebook for this tutorial here.

Here is the original post:

Streamline Data Pipelines: How to Use WhyLogs with PySpark for Data Profiling and Validation - Towards Data Science

Employers, ditch the AI detection tools and ask one important question instead.

I saw a post on LinkedIn from the Director of a Consulting Firm describing how he assigned an essay about model drift in machine learning systems to screen potential candidates.

Then, based on criteria that he established based on his intuitions (you can smell it) he used four different AI detectors to confirm that the applicants used ChatGPT to write their responses to the essay.

The criteria for suspected bot-generated essays were:

One criteria notably missing: accuracy.

The rationale behind this is that using AI tools is trying to subvert the candidate selection process. Needless to say, the comments are wild (and very LinkedIn-core).

I can appreciate that argument, even though I find his methodology less than rigorous. It seems like he wanted to avoid candidates who would copy and paste a response directly from ChatGPT without scrutiny.

However, I think this post raises an interesting question that we as a society need to explore is using an LLM to help you write cheating during the hiring process?

I would say it is not. Here is the argument for why using an LLM to help you write is just fine and why it should not exclude you as a candidate.

As a bonus for the Director, Ill include a better methodology for filtering candidates based on how they use LLMs and AI tools.

Excerpt from:

Does Using an LLM During the Hiring Process Make You a Fraud as a Candidate? - Towards Data Science