Can AI Machine Research Fashion Overcome Biased Data Sets
Unsupervised artificial intelligence (AI) models that automatically discover hidden patterns in natural language datasets capture linguistic regularities that reflect human biases, such as racism, sexism, and ableism.1 These unsupervised AI models, namely word embeddings, provide the foundational, general-purpose, numeric representation of language for machines to process textual data.
Can AI machine research fashion overcome biased data sets
Social media platforms automatically decide which users should be exposed to certain types of content present in political advertisements and information influence operations, based on personality characteristics predicted from their data.5 As researchers identify and measure the harmful side effects of NLP algorithms that incorporate biased models of language, regulation of algorithms and AI models can help alleviate the harmful downstream impacts of large-scale AI technologies.
To analyze these natural and artificial decision-making processes, proprietary biased AI algorithms and their training datasets that are not available to the public need to be transparently standardized, audited, and regulated. Technology companies, governments, and other powerful entities cannot be expected to self-regulate in this computational context since evaluation criteria, such as fairness, can be represented in numerous ways. Satisfying fairness criteria in one context can discriminate against certain social groups in another context. Moreover, with new AI techniques, desired fairness criteria can be artificially satisfied, while discriminating against minority populations, by applying AI tricks via adversarial machine learning.14 Meanwhile, it might take centuries to develop sophisticated AI technologies aligned with human values that can self-regulate.
Biased NLP algorithms cause instant negative effect on society by discriminating against certain social groups and shaping the biased associations of individuals through the media they are exposed to. Moreover, in the long-term, these biases magnify the disparity among social groups in numerous aspects of our social fabric including the workforce, education, economy, health, law, and politics. Diversifying the pool of AI talent can contribute to value sensitive design and curating higher quality training sets representative of social groups and their needs. Humans in the loop can test and audit each component in the AI lifecycle to prevent bias from propagating to decisions about individuals and society, including data-driven policy making. Achieving trustworthy AI would require companies and agencies to meet standards, and pass the evaluations of third-party quality and fairness checks before employing AI in decision-making.
Without access to the training data and dynamic word embeddings, studying the harmful side-effects of these models is not possible. And having access to word embeddings and data can facilitate new scientific discoveries for social good, including advances such as the discovery of new materials from word embeddings.17 However, developers of large language models are unable to share the training corpora due to data privacy laws. Moreover, adversarial machine learning researchers recently showed that it is possible to extract training data, including personally identifiable information, from large language models.18 Researchers, developers, and policymakers desperately need an environment to work on these models together, however, the lack of established standards hinders scientific progress and is highly likely to damage society. Passing federal privacy legislation to hold technology companies responsible for mass surveillance is a starting point to address some of these problems. Defining and declaring data collection strategies, usage, dissemination, and the value of personal data to the public would raise awareness while contributing to safer AI.
The complex AI bias lifecycle has emerged in the last decade with the explosion of social data, computational power, and AI algorithms. Human biases are reflected to sociotechnical systems and accurately learned by NLP models via the biased language humans use. These statistical systems learn historical patterns that contain biases and injustices, and replicate them in their applications. NLP models that are products of our linguistic data as well as all kinds of information that circulates on the internet make critical decisions about our lives and consequently shape both our futures and society. These NLP models are behind every technology using text such as resume screening, university admissions, essay grading, voice assistants, the internet, social media recommendations, dating applications, news article summarizations, machine translation, and text generation. If these new developments in AI and NLP are not standardized, audited, and regulated in a decentralized fashion, we cannot uncover or eliminate the harmful side effects of AI bias as well as its long-term influence on our values and opinions. Undoing the large-scale and long-term damage of AI on society would require enormous efforts compared to acting now to design the appropriate AI regulation policy.
Transfer Learning [10, 11] is another interesting paradigm to prevent overfitting. Transfer Learning works by training a network on a big dataset such as ImageNet [12] and then using those weights as the initial weights in a new classification task. Typically, just the weights in convolutional layers are copied, rather than the entire network including fully-connected layers. This is very effective since many image datasets share low-level spatial characteristics that are better learned with big data. Understanding the relationship between transferred data domains is an ongoing research task [13]. Yosinski et al. [14] find that transferability is negatively affected primarily by the specialization of higher layer neurons and difficulties with splitting co-adapted neurons.
It is a generally accepted notion that bigger datasets result in better Deep Learning models [23, 24]. However, assembling enormous datasets can be a very daunting task due to the manual effort of collecting and labeling data. Limited datasets is an especially prevalent challenge in medical image analysis. Given big data, deep convolutional networks have been shown to be very powerful for medical image analysis tasks such as skin lesion classification as demonstrated by Esteva et al. [25]. This has inspired the use of CNNs on medical image analysis tasks [26] such as liver lesion classification, brain scan analysis, continued research in skin lesion classification, and more. Many of the images studied are derived from computerized tomography (CT) and magnetic resonance imaging (MRI) scans, both of which are expensive and labor-intensive to collect. It is especially difficult to build big medical image datasets due to the rarity of diseases, patient privacy, the requirement of medical experts for labeling, and the expense and manual effort needed to conduct medical imaging processes. These obstacles have led to many studies on image Data Augmentation, especially GAN-based oversampling, from the application perspective of medical image classification.
Many studies on the effectiveness of Data Augmentation utilize popular academic image datasets to benchmark results. These datasets include MNIST hand written digit recognition, CIFAR-10/100, ImageNet, tiny-imagenet-200, SVHN (street view house numbers), Caltech-101/256, MIT places, MIT-Adobe 5K dataset, Pascal VOC, and Stanford Cars. The datasets most frequently discussed are CIFAR-10, CIFAR-100, and ImageNet. The expansion of open-source datasets has given researchers a wide variety of cases to compare performance results of Data Augmentation techniques. Most of these datasets such as ImageNet would be classified as big data. Many experiments constrain themselves to a subset of the dataset to simulate limited data problems.
Data Augmentation cannot overcome all biases present in a small dataset. For example, in a dog breed classification task, if there are only bulldogs and no instances of golden retrievers, no augmentation method discussed, from SamplePairing to AutoAugment to GANs, will create a golden retriever. However, several forms of biases such as lighting, occlusion, scale, background, and many more are preventable or at least dramatically lessened with Data Augmentation. Overfitting is generally not as much of an issue with access to big data. Data Augmentation prevents overfitting by modifying limited datasets to possess the characteristics of big data.
Unsupervised machine learning algorithms do not require data to be labeled. They sift through unlabeled data to look for patterns that can be used to group data points into subsets. Most types of deep learning, including neural networks, are unsupervised algorithms. Unsupervised learning algorithms are good for the following tasks:
There is also the problem of machine learning bias. Algorithms trained on data sets that exclude certain populations or contain errors can lead to inaccurate models of the world that, at best, fail and, at worst, are discriminatory. When an enterprise bases core business processes on biased models it can run into regulatory and reputational harm.
Generating unbiased data can be easier than expensive data mining. Gathering actual data requires measuring, interviewing, a large sample size, and, in any case, plenty of work. AI-generated data is cheap, and it only requires data science/machine learning techniques.
Machine learning and data mining often employ the same methods and overlap significantly, but while machine learning focuses on prediction, based on known properties learned from the training data, data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases). Data mining uses many machine learning methods, but with different goals; on the other hand, machine learning also employs data mining methods as "unsupervised learning" or as a preprocessing step to improve learner accuracy. Much of the confusion between these two research communities (which do often have separate conferences and separate journals, ECML PKDD being a major exception) comes from the basic assumptions they work with: in machine learning, performance is usually evaluated with respect to the ability to reproduce known knowledge, while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge. Evaluated with respect to known knowledge, an uninformed (unsupervised) method will easily be outperformed by other supervised methods, while in a typical KDD task, supervised methods cannot be used due to the unavailability of training data. 041b061a72