Artificial Intelligence – The Revolution Hasn’t Happened Yet (2018)
medium.comThe problem had to do not just with data analysis per se, but with what database researchers call “provenance” — broadly, where did data arise, what inferences were drawn from the data, and how relevant are those inferences to the present situation? While a trained human might be able to work all of this out on a case-by-case basis, the issue was that of designing a planetary-scale medical system that could do this without the need for such detailed human oversight.
I'm not a data scientist and I've never encountered that term "provenance" before but I've encountered the problem he talks about in the wild here and there and have searched for a good way to describe it. His ultrasound example is a great, chilling, example of it.
I also like the term "Intelligence Augmentation" (IA). I've worked for a couple companies who liberally sprinkled the term AI in their marketing content. I always rolled my eyes when I came across it or it came up in say a job interview. What we were really doing, more practically and valuably, was this: IA through II (Intelligent Infrastructure), where the Intelligent Infrastructure was little more than a web view on a database that was previously obscured or somewhat arbitrarily constrained to one or two users.
The IA terminology brings to mind the classic "Augmenting Human Intellect"[1] essay by Doug Engelbart (famous for giving "The Mother of all Demos"[2])
It reminds me of the memex essay: https://www.theatlantic.com/magazine/archive/1945/07/as-we-m...
Also the article by Ted Nelson, "As We Will Think": http://teleshuttle.com/ftp/TedNelsonDocs/AsWeWillThink1972[R...
https://medium.com/@rreisman/as-we-will-think-the-legacy-of-...
Provenance is an idea that shows up in multiple fields. I first encountered it in discussions of archeology. But then it showed up in, for example, https://www.ralfj.de/blog/2020/12/14/provenance.html discussing how improper handling of pointer provenance can cause code to get miscompiled.
https://en.wikipedia.org/wiki/Provenance gives more on the term and the way it shows up.
You'll hear the term provenance used quite a bit on PBS's long running Antiques Roadshow.
Provenance is also used in wine and art where a chain of custody, which the value largely hinges on, must be through trustworthy people or institutions.
More interestingly, both wine and art have had their provenance hinges widely exploited for massive profit while posh people think they're enjoying something exclusive.
This is how I came to know the term. I was a fan of the PBS weekend lineups during the 90s.
Data provenance is a standard term of art in machine learning and data science, a “data 101” kind of thing, with many OSS and vendor tools built up to solve provenance problems, like DVC, Pachyderm, kubeflow, mlflow, neptune, etc.
worked with stats, machine learning and data science for 10+ years now. never heard the term used until now. (that's not to say I'm not familiar with the things the term refers to, indeed, most of the intellectual frameworks I've worked with break each of the things that make up provenance into far more fine grained concepts).
course, I've also never heard of or touched the software you listed there either, but that may be because I don't view the data science and machine learning I'm interested in as being about specific software or vendor software...
sounds more database- lingo to me...
It's a common term used in data governance. It's found less in the academic literature, and more in software demos and vendor brochures. You'll also hear "data lineage", which is the context in which the term arises.
"Provenance" just means where the data came from. [1]
It's one of those shibboleths and terms of art used by people in industry. If you go to trade-shows you'll hear it being used -- it's worth knowing if nothing else but for its sociological value among the data software tools crowd.
Side: it's a little like the word "inference" being used as a verb by folks in AI (example usage: we use GPUs to speed up model "inferencing") -- in AI, to inference means to "predict". It's a term of art. If someone with a traditional statistics background went to a deep learning conference, they are likely to be very confused because in traditional statistics, inference means to obtain parameters θ in a model y = f(x,θ), whereas in AI, inferencing refers to obtaining y.
[1] https://en.wikipedia.org/wiki/Data_lineage#Data_provenance
I've also worked as a data scientist for a few years and have never heard or used the word "provenance" in a DS context. Some people used it in the oil & gas industry when talking about where reservoir sands came from, but that usually garnered a eye-roll and mental translation to more everyday language.
Regardless of the term chosen, the concept of 'provenance' described here is the essential purpose behind the scientific notebooks used daily by experimentalists in industry and academia. Without thoroughly recording the bases for your experiment it almost surely will not be reproducible.
Where I work, (a large pharmaceutical), these notebooks are taken very seriously by biologists, chemists, and chemical engineers, and increasingly are shaping the mindset of our data scientists (who have yet to adopt them).
Given the longstanding practice of documenting experiment design and method, I think it's probably long overdue that the exploratory analysis of experiment-based data must also adopt more rigorous governance to ensure that necessity and sufficiency are ensured when drawing inferences from experiment, especially when the data used was not originally intended to answer the current question posed.
It’s shocking if you’ve worked professionally in statistics and not heard about data provenance.
A few publications from ~2011-2015 period:
http://ceur-ws.org/Vol-1558/paper37.pdf
https://ieeexplore.ieee.org/document/5739644
https://link.springer.com/chapter/10.1007/978-3-642-53974-9_...
Add a variety of additional links dating back a bit further (note the emphasis in this case on research data and tracking state of an experiment).
https://nnlm.gov/data/thesaurus/data-provenance
Data provenance is not a database / data warehouse term. It is uniquely and specifically a basic “101” concept of statistical science and ML / data science, where the custody and tracking of data are specifically tied to iterations of experiments, prototypes and research, for the sake of reproducibility.
If I was interviewing an experienced statistical researcher and they didn’t at least have a working knowledge of the core concepts, that would be a huge red flag.
I'm not saying it doesn't exist, but I am saying it must be jargon used within a particular community or minority subset of general stats/ machine learning/AI. honestly, I still think it's a database/ enterprise term because I've worked for our national statistics office and never heard it in the statistics community either. I have frequently heard data lineage however, but again, that's a database/ enterprise type person lingo: when people use that word I know immediately the background they're coming from.
Another poster mentioned vendor brochures and trade shows, which is in line with my expectations about which community it stems from, and also explains why I've never heard of it because I try to keep away from such environments these days.
Everywhere I've been the things which I take to make up "provenance" have generally been referred to under the simple label of "data quality", with separate subset definitions and measures such as timeliness, source, authority, format, history, suitability, verification, etc.
Of course, that's assuming people even worry about such things. In practice, let's be frank, anyone who's worked with data science knows they actually get shorter shrift than they deserve in practice: I'm probably among a minority of people in the real world who actually take things seriously, and I find myself on a constant crusade to remind people that just because a data point exists in a data set doesn't mean it's useful/ appropriate/ truthful/ unbiased.
data quality is a bit problematic, because I can see it being used by people who think provenance doesn't have any thing to do with quality, and from a variety of fields, but it is also infinitely more popular according to historical search trends, and in my last three jobs provenance would fall under the data quality framework.
It’s very, very widely used jargon. I’d put “data provenance” on par with “overfitting” or “GPU model training” in terms of the high, ubiquitous place it occupies in mainstream machine learning.
Sorry, I have to disagree here. Its a term of art in some of the literature, but it's definitely not that widespread, certainly not in consumer tech data science, where I work.
I’ve worked professionally in quant finance, image processing, defense research, and several mid-to-large ecommerce and payment processor companies.
In all of them, data provenance has been a first class consideration of machine learning and data platform teams, like a day-to-day concern and baked in to architecture review guidelines and production checklists and whatnot for every ML project.
In many of these companies we had teams of 20-40 ML scientists, all of whom knew about data provenance as a first class consideration in their work, had experience with it from their past jobs and academic programs, and considered it on equal footing with any aspect of data curation, model selection, model training and model serving.
I mean, I shouldn't be surprised, as given our previous interactions, I feel like you are the anti-me, in that our experiences of similar things is so wildly divergent.
Shrug, such is life I guess. That being said, I care deeply about this stuff (but didn't have a word), so perhaps it will be easier to convince people to pay attention to the data with said word.
TIL.
I’ve worked as a data engineer for the last two years and never heard of this being used in this context before.
Typically the word “data lineage” is used to mean this in my experience.
I don’t think I’ve ever been in a meeting where someone mentioned provenance except referring to a show about paintings.
Lineage isnt the same thing, being a more specific technical term referring to keeping the history of datasets and where they came from (basically), but people actually say the words “data governance” and “lineage”.
Another important use of data provenance is in GDPR. You have to be able to know the source of each data you use and be able to remove them from storage and backups at request.
Wikipedia says "Provenance is conceptually comparable to the legal term chain of custody." https://en.wikipedia.org/wiki/Provenance
If you (ever) need to update your data, you need to know where you got it from, what was wrong with it originally, and how to pull it again.
Provenance, as a concept and specification, is well established in digital domain, as described by W3C's PROV specification https://www.w3.org/TR/prov-overview/ Ability to trace, audit, and reproduce artifacts or processes are some applications of provenance that align with needs for explainability in data analytics and data science/AI (XAI).
We address the problem of adding provenance without rewriting your tensorflow/scikit-learn/pytorch/pyspark application by adding CDC support in the ML stack and collecting all events in a metadata layer, building an implicit provenance graph. It's now part of the open-source Hopsworks platform. See this USENIX OpML'20 talk on it: https://www.youtube.com/watch?v=PAzEyeWItH4
It's weird to me that people build libraries on top of the ML stack to track provenance, when it's really the ML library's job to do that for its inputs. However it is a right pain building it into the ML library as it affects all the interfaces. We build data, model & evaluation provenance objects into our ML library, Tribuo (https://tribuo.org), as a first class part of the library. You can take a provenance and emit a configuration to rerun an experiment just by querying the model object. It is built in Java though, which makes it a little easier to enforce the immutability and type safety you need in a provenance system.
edit: I should add that I'm definitely in favour of having provenance in ML systems, and libraries layered on top are the way that people currently do that. It's just odd that people aren't working on adding that support directly into scikit-learn/TF/pytorch etc.
MLFlow and TFX try to add some form of provenance by polluting your code with "logging" calls. A good thing MLFlow has added is auto-loggers - we also added them in our Maggy framework ( https://www.logicalclocks.com/blog/unifying-single-host-and-... ).
I totally agree that where you have framework hooks, you should have provenance, but given there's no standard for what provenance is, no defacto open-source platform, the sklearn and tf and pytorch folks rightly steer clear. We see that if you have a shared file system, you can use conventions for path names (features go in 'featurestore', training data in 'training', models in 'models', etc), to capture a ton of provenance data.
I first encountered the term "provenance" in 2007 when I was working on an undergraduate research project at UC Santa Cruz in the area of metadata-based search for the Ceph distributed file system. I particularly remember reading this USENIX ATC 2007 paper: https://www.usenix.org/conference/2007-usenix-annual-technic.... This was my introduction to the concept of provenance.
Professor Margo Seltzer (https://www.seltzer.com/margo/) is a well-known researcher in the area of provenance. I highly recommend reading her papers if you're interested, starting with her USENIX ATC 2006 paper "Provenance-Aware Storage Systems".
"Intelligence Augmentation" makes good PR, like those old Apple commercials, "Be All You Can Be" (no, that's the US Army; Apple's was "The Power To Do Your Best.")
But the money is in replacing humans.
They may sound a bit cheeky, but those humans are us. I was happy when a client had asked something and I realised I didn't have to do it because our machine learning platform had already what was requested.
We had built it precisely to free us from certain repetitive things in machine learning projects [environment set up, near real-time collaboration on notebooks, scheduling long-running notebooks, experiment tracking, model deployment and monitoring]. We used to scramble and do all that, request help from our colleagues and pull them from what they were doing. This was really taxing and bad for morale, jumping around from one context to another.
I had a huge smile contemplating all the work I was about to not do.
There are many things where the humans themselves ought to be "augmented". Case in point, in some projects involving predictive maintenance, the stakes of an incident can be around $100MM and all these processes depend on a human being alert at all times during their very long shift, with a bunch of other things happening simultaneously. This is very stressful and these people actually want to be "augmented". They want something to help them and catch things they would have missed because they haven't had proper sleep or were too busy solving another urgent and important problem. It is the people themselves who come to us and ask us for our help to help them solve these problems.
It may sound cheeky, and in many cases at many companies it is cheeky and it is PR like saying "partners" instead of "drivers", or "dashers" instead of "delivery person". In some cases it really is what happens. At least from my biased perspective with the actual humans who were asking for "augmentation" to do their job.
Which is true, but those pesky humans, it turns out that really replacing them is soooo tricky!! They end up hanging around in the business process spending money and whining about how evil you are, and all the time your competitors are bolstering their employees capabilities making then happier and more productive and pushing wishy washy messages of corporate social responsibility at your customers.
I think the goal of all technology is not only to reduce cost (replace humans) but to improve the job they do — make them faster, stronger, smarter. IMO the enhancement part of the tech evolution equation has been much underconsidered.
Any time a tool evolves, it changes not only how a task is done but also why. In the case of implementing a business process, the revision process is best served by reconsidering why what is done now and taking the opportunity to evolve the old role into making a richer contribution that introduces a new and improved path through the problem space.
That's IA, and IMO, it's the Great White Hope that AI might yet lead to a future world that engages humans more rather than the default dystopia where we're all redundant and irrelevant.
I encountered term provenance when I was learning Apache Nifi.
The brittleness of mainstream ML to out of distribution data is one of the most fundamental channels for error. There are very few domains where a static environment can be depended on over the long term. If machine learning is to be approached as an engineering discipline there will need to be practices established for validating models throughout their life cycle. One potential resource that can support this type of systematic evaluation is the Automunge open source library for assembling data pipelines, which has automatic support for evaluating data property drift in feature sets serving as basis for a model. (disclosure I am founder of Automunge)
This post should (1) reflect the 2018 posting date, and (2) the main hosting site: https://hdsr.mitpress.mit.edu/pub/wot7mkc1/release/9
I've added 2018 above. Thanks!
That URL doesn't seem to be the original source though.
How would one put it?
"Adaptive Intelligence" might be described as the ability to be given a few instructions, gather some information and take actions that accomplish the instructions. It's what "underlings", "minions" do.
But if we look at deep learning, it's almost the opposite of this. Deep learning begins with an existing stream of data, a huge stream, large enough that the system can just extrapolate what's in the data, include data leads to what judgements. And that works for categorization and decision making the duplicates what decisions humans make or even duplicates what works, what wins in a complex interaction process. But all that doesn't involve any amount of adaptive intelligence. It "generalizes" something but our data scientists have no idea exactly what.
The article proposes an "engineering" paradigm as an alternative to the present "intelligence" paradigm. That seems more sensible, yes. But I'm doubtful this could accepted. Neural network AI seems like a supplement to the ideology of unlimited data collection. If you put a limit on what "AI" should do, you'll put a limit on the benefits of "big data".
Neural nets don't generalize much, they interpolate between training examples. But if you couple them with search (MCTS) then you can do logic and reasoning with them, like AlphaGo.
You can also put any algorithm you want inside a neural net as long as you have a mechanism to pass gradients back - for example in the final layer you could have a complex graph-matching algorithm to map the predictions to the target, or you could put an ODE solver as a layer, or a logic engine, or a database.
Dead link for me, but archive.org has a snapshot: https://web.archive.org/web/20201224185231/https://rise.cs.b...
We are chasing the wrong things. Our conceptualization of the problem domain is fundamentally insufficient. Even if we took our current state of the art and scaled it up 1,000,000x, we are still missing entire aspects of intelligence.
The AI revolution is very likely something that will require a fundamental reset of our understanding of the problem domain. We need to identify a way to attack the problem in such a way that we can incrementally scale all aspects of intelligence.
The only paradigm that I am aware of which seems to hint parts of the incremental intelligence concept would be the relational calculus (aka SQL). If you think very abstractly about what a relational modeling paradigm accomplishes, it might be able to provide the foundation for a very powerful artificial intelligence. Assuming your domain data is perfectly normalized, SQL is capable of exploring the global space of functions as they pertain to the types. This declarative+functional+relational interface into arbitrary datasets would be an excellent "lower brain", providing a persistence & functional layer. Then you could throw a neural network on top of this to provide DSP capabilities in and out (ML is just fancy multidimensional DSP).
If you know SQL you can do a lot of damage. Even if you aren't a data scientist or have a farm of Nvidia GPUs, you can still write ridiculously powerful queries against domain data and receive powerful output almost instantaneously. The devil is in the modeling details. You need to normalize everything very strictly. 20-30 dimensions of data derived into a go/no-go decision can be written in the same # of lines of SQL if the schema is good. How hard would this be on the best-case ML setup? Why can't we just make the ML write the SQL? How hard would it be for this arrangement to alter its own schema over time autonomously?
You're talking about logic - SQL is basically a "logic language", it's just not entirely evident.
Logic programming was the AI paradigm for more or less most of the 20th century and has fallen out of favor.
Many people have talked about combining the neural net/extrapolation/brute-force approach with the logic approach. That hasn't born fluid yet but who knows.
There is somewhat renewed interest in hybrid approach. See, for example, DeepProbLog[1][2][3] - a combination of Deep Learning and probabilistic logic.
[1] https://arxiv.org/abs/1805.10872
I don't think there ever has not been interest in hybrid approaches - I think each I've looked over ten or more years, there was at least one hybrid thing (Neural Turing Machines comes to mind). I think the problem is no one has figured out a way to make them "work".
Or not even that they don't function but you need a way to demonstrate that such things are "really good", that they solve real problems that neither "business logic system" (the real existing remnant of GOFAI) nor neural networks can solve. And the key both logic systems and neural networks have is how they pretty standardized. logic systems are like regular programming and neural networks have their train/test/verify cycle understood (and even with that, they're probably overused/misused at this point given the hype).
BPE as used in NLP might count as a successful hybrid approach. Maybe the hybrid approaches that work out will be really specific purpose like that for a while.
Bullshit. Write me a better cat detector in SQL, or protein folder, or super-human board game player. And, by the way, ML does write SQL [1].
Deep learning has all these disadvantages and difficulties because we moved the goalposts too many times and now want so much more out of it than regular software. A model has to be accurate, but also unbiased, updated timely and explainable and verified in much detail; also efficient in terms of energy, data, memory, time and reuse in other related tasks (fine-tuning).
We already want from an AI what even an average human can't do, especially the bias part - all humans are biased, but models must be better. And recently models have been made responsible with fixing societal issues as well so they've become a political battleground for various factions with different values - see recent SJW scandals at Google.
With such expectations it's easy to pile on ML but ML is just a tool under active development, with practical limitations, while human problems and expectations are unbounded.
[1] a neural SQL patent: https://patentimages.storage.googleapis.com/af/78/be/92ee342...
> We are chasing the wrong things...
It isn't obvious we are chasing anything. The graphics card industry was chasing more performance and the field of AI research was pushed along by that.
The field is full of smart people doing impressive work but there havn't ben any fundamental breakthroughs that aren't hardware driven.
It's not all possible because we have more compute. Algorithms have also gotten better and more efficient, on a range of 10-50x.
If you follow John Carmack’s foray into AI then it appears he’s also approaching it from the “scaling compute” perspective.
I think the inhibitor to scale is actually model compression. You’re right that scaling up 1Mx won’t cut it. That’s because fidelity is still too high. We already know the brain is a very efficient machine for storing heuristics and compressed models. Also related to why it’s prone to err. Information theory is the right framework here imo. Other related concerns: hierarchal organization of information and model comparison.
> Our conceptualization of the problem domain is fundamentally insufficient.
There have been people in the AI community saying this since at least the 80s/90s (e.g. Hofstadter). It's an old idea that has been difficult to get much traction on, partially because it's a long way from applications. NN, SVN, etc. for all of their limitations can draw that line pretty easily.
What is DSP?
Digital signal processing maybe?
Correct, in this context I meant Digital Signal Processing.
> We are chasing the wrong things. Our conceptualization of the problem domain is fundamentally insufficient. What we really need is GOFAI.
Back at ya mate :D.
Actually I think the first example was a really simple case, where statistics would expose the error. So even the doctor said, that they experienced an uptick in Down syndrome diagnoses. So basically they just didn't investigate it properly. From my experience every advanced ML-System have proper monitoring and such anomalies would be detected very fast. Especially when you change the machines. Actually it is a shame that the doctors couldn't figure it out by themselves or at least investigate it properly.
Yeah, it's baffling that there was no one there to link the new machine with the uptick in the diagnoses. It's almost like they didn't care at all that there was an increased number of diagnoses.
Cross posted medium link: https://medium.com/@mijordan3/artificial-intelligence-the-re...
Since the original URL (https://rise.cs.berkeley.edu/blog/michael-i-jordan-artificia...) is responding slowly and points to the medium.com URL as the original source anyhow, we've changed to the latter. Thanks!
>>> in Down syndrome diagnoses a few years ago; it’s when the new machine arrived
Hang on - uptick in diagnosis (ie post amniocentesis) or uptick in indicators. One indicates unnecessary procedures, one indicates a large population of previously undiagnosed downs ....
One assumes the indicator - and greatly hope there is improved detection as I had at least one of these scares with my own kids
More false positives from ultrasounds could lead to more amniocentesis true positives simply by increasing the number of amniocentesis performed. Without more information it's not possible to tell.
But since more babies should be born without DS, despite more diagnoses of DS due to more tests, shouldn't the negatives also increase (more) after the amniocentesis? I expect both positives and negatives to increase with the increase in negatives being more than the increase in positives.
Thank you that's my point better put. Unless they uncover a hidden pool of Downs cases (unlikely) this was an increase in false positives leading to an increase in amniocentesis leading to an increase (1/300) of preventable foetal deaths. I think.
I was careful to say true positives rather than true positive rate. This is why without more information it is hard to know exactly what is going on.
If you get a true positive from amniocentesis, the positive ultrasound would have been a true positive too, though, right?
Presumably what he is leaving out is that the increase in white-spots led to more amniocentesis, which then confirms the Down syndrome. If you did amniocentesis on all babies, it would of course increase the diagnosis rate even more.
Whether this is a bad thing, as he claims, depends on whether you believe screening was being done optimally before, and that will depend quite a bit on things left out like the utility of not having a Down baby. (He doesn't present his working out the entire scenario, as it's just an aside, but hopefully before Jordan went around telling people how to change their prenatal screening systems, he did work it out a little bit more than back-of-the-envelope.)
Actually you would not expect it to increase the Downs rate at all - the null hypothesis is that there are X% Downs babies born and Y% identified through amniocentesis (where X-Y is z% the percentage of parents choosing termination)
edit: actually there is a Zt (percent of parents choosing termination after detection) and Zu (percent age of undetected cases going to term). Zt is a social / moral thing and won't change based on better pixel resolution, but Zu should not be expected to change either - we are assuming there has been no change to the real rate of Downs (which requires something else) and no change to rate of parents choosing termination (see morals)
so ...
If Y% increases a lot (better detection of an underlying true rate) then either X% must increase or z% must. Neither of which i think we expect or know about.
So what I hope happened was dramatically better training for operators on the new ultrasound (kind of like exactly what did not happen onthe 747Max)
So either the OP was one of the first to spot this issue, and tipped off the whole medical industry, or, and this is where my money goes, he followed the reasoning of dozens of professionals who were several years ahead of him (naturally) and was reassured by someone who just saw "anxious parent" in front of him.
But that's fine too :-)
"Actually you would not expect it to increase the Downs rate at all - the null hypothesis is that there are X% Downs babies born and Y% identified through amniocentesis (where X-Y is z% the percentage of parents choosing termination)"
No, see, you're misunderstanding and making the exact same mistake by assuming that the nurses sat down and crunched the spreadsheets, which they obviously did not, and implicitly conditioning. Yes, if you slice the multivariable data just the right way to extract the conditional %, the increase would be the same. Screening more women won't make the conditional percentage go up.
However, if you are just a nurse, observing # of women coming in, and # of confirmed Down babies coming out... Screening more women will mechanically make the # of confirmed Down babies go up. And that's what the rate is, it's a count.
What I miss most,in discussions about AI, is the motivation factor,which is the driving force behind every single thing we humans do. How can we create a system that would be motivated to evolve in order to better itself. Humans created all sorts of things because fear,hunger,or pleasure was so strong and it couldn't be pushed away. What will happen to an AI powered robot that one day decide that going into radioactive areas isn't quite what it wants and will say 'screw it'?
The issue is that intelligence by itself provides to motivations or objectives. Intelligence is simply a tool to be used in order to get things done. In humans the objectives are provided by instincts, emotions and biological needs.
In AI we will have to provide the goals, but as the paper clip maximiser thought experiment shows, we’re going to have to be very careful and thoughtful about it.
Part of the challenge of pursuing this comprehensive type of AI infrastructure is that it requires massive coordination and collaboration. Unfortunately the incentives in both industry and academia make it difficult to even start such a project. As a result we're stuck with incremental work on narrow problems.
I've been on both sides of table (started in industry developing AI solutions and now in academia pursuing phd in AI). When I was on the industry side, where the information and infrastructure was there to build such a system, you had to deal with the bureaucracy and institutional politics.
In academia, the incentives are aligned for individual production of knowledge (publishing). The academic work focuses on small defined end-to-end problems that are amenable to deep learning and machine learning. The types of AI models that emerge are specific models solving specific problems (NLP, vision, play go, etc).
It seems to move towards developing large AI systems we need a model of new collaboration. There are existing models in the world of astrophysics and medical research that we can look to for inspiration. Granted they have they have their own issues of politics but it's interesting that similar scope projects haven't emerged on the AI side yet.
The incentive structure that seems clearly best (though not greatly) suited to this large-scale intelligence infrastructure is public investment in publicly owned systems.
Jordan seems to maybe gesture at this, as who owns all the bridges in the the USA? Governments. If we are talking “societal-scale medical system” a majority of people would want that publicly owned and operated and universally accessible.
We’ve already seen in industry that the incentives are to massively in favour of creating walled-gardens that lock in users and thus profits. No societal-wide system should work like our social media ecosystem (FB, Snapchat, TikTok). The dominant profit incentives are also not “human-centric”, as Jordan constantly emphasises. Well, they’re only so if we assume profit-making activity is tightly aligned with “human-centric” concerns. Some will say yes, but to me our climate disaster and the USA mass incarceration system are strong enough evidence that the answer is no.
I think some wealthy Northern European countries are setup well enough to produce “Intelligent Infrastructure”, except for the fact that most of the talent is in the USA.
If the rising trend toward putting the giant info-broker corporations on tighter leashes continues, I'm hopeful that we might finally start to discuss the 'civil rights of data' — who actually owns data of all kinds, but especially personal data, and what obligations come with using it, sharing it and ensuring that it not be abused.
Jordan argues that leaps in human-imitative AI are not necessary to solve IA/II problems -- "We need to solve IA and II problems on their own merits, not as a mere corollary to a human-imitative AI agenda."
However, achieving near-human level accuracy on tasks such as classifying images of cars or road signs would be immensely useful to the proposed II-type system that handles large-scale self-driving transportation (individual cars would conceivably need the ability to understand their local environments and communicate this to the overall network).
I agree with his argument that there should be a shift in the way we think about problems in "AI", but I don't think we should necessarily think that progress in human-imitative AI problems and IA/II problems are mutually exclusive.
This sounds like the longstanding debate between weak / narrow vs strong AI. Can improving the former make progress toward the latter? I'm inclined to agree with Jordan that we shouldn't expect the two to enhance the other much less commingle. Just as advancement of one classical algorithm rarely enhances another, I think it's unlikely the next generation of object recognition is going to advance speech recognition or reading comprehension.
Probably more essentially, until AI escapes its current dependency on pattern matching driven solely by accumulation of probabilistic events, I see little chance that human-level general-purpose cognition will arise from our current bases for AI, namely observing innumerable games of chess or watching millions of cars wander city streets.
What i have seen in the field was a frenzy of doing ML. What happened was that first of all companies needed to understand what ML was. Then understand the tools available. Once you start exploring the tools you will find that at every stage of a ML pipeline there is 2 or more different ways of doing things. S3/GCS, BigQuery, Spark, Beam, TensorFlow, Pytorch, Google, Azure, Amazon, notebooks, Jupyter, JupyterHub..KubeFlow, TFX...etc. okay you pick the tools needed, then you need to put them together...and hire people...that's challenging. I believe we need to wait for AutoML pipelines from data análisis to.prediction to start seeing really advancement in production systems.
> IA will also remain quite essential, because for the foreseeable future, computers will not be able to match humans in their ability to reason abstractly about real-world situations.
I broadly agree with what this article says, but depending how you define "foreseeable future" I find this to be a dangerously naive viewpoint that just assumes nothing will change quickly.
I'm not stupid enough to say abstract reasoning about the real world is a simple problem or right around the corner, but there's no evidence so far to indicate it's much further off than, say, object recognition was when Minsky (or more likely Papert, apparently?) assigned it as an undergrad project. We pour exponentially more money into research each year, and have more and better hardware to run it on. We're going to hit the ceiling soon re: power consumption, sure, but some libraries are starting to take spiking hardware seriously which will open things up a few orders of magnitude. There are dozens of proposed neural architectures which could do the trick theoretically, they're just way too small right now (similar to how useless backprop was when it was invented).
Are we a Manhattan Project or three away from it? Sure. That's not nothing, but we're also pouring so much money into the boring and immediately commercializable parts of the field (all the narrow perception-level and GAN can that NeurIPS is gunked up with) that if any meaningful part of that shifted to the bigger problems, we'd see much faster progress. That will happen in a massive way once someone does for reasoning what transformers did for text prediction: just show that it's tractable.
Looks like we have reached a point where we see slower growth & innovation in tech in coming decade. AI was supposed to be the next big disruptor but my guess is we will just see minor progress in automation and far away from anything disruptive. Singularity might may not even be possible in next century
Discussed at the time: https://news.ycombinator.com/item?id=16873778
So what do we name this new emerging engineering discipline?
AI engineering?
Cybernetic engineering?
Data engineering?
Cybernetics and systems engineering certainly has to make a comeback if we are to solve coordination problems like this at planetary scale. It deeply saddens me that we almost reached a popular acceptance of cybernetics in the 60s, but it passed us by - we'd be in a much better position now if it had become a mainstream science in the way that other, much less useful sciences have.
I agree.
As I see from reading a little about the field's history and the literature, it suffered the same fate of other endeavors that are complex and still have a lot to be solved.
people become interested in it, try to find simpler 'popular' formulation and then the watered down versions become more popular than the original more complex version that need more rigor and discipline.
the watered down versions become more popular but without the rigor and discipline, you can argue and conclude everything and they opposite with these tools.
so people on the outside see the field as yet another fad and the whole field die down taking down with it the original version.
much like in AI with everyone labeling their stuff as AI which dilute the term more and more as time passes.
what Cybernetics and systems engineering needs is a rebranding and separation from the more 'soft' side that developed latter.
this is where I think some researchers on category theory like Jules Hedges might help. it would help defining dynamical and more general system in a vague but still formal way, say with a computer proof assistant sort of tool.
Systems theory is the rebrand of cybernetics IMHO. Cybernetics got diluted as a concept sure to its association with technology and scifi - AI, the Internet, and cyberpunk fiction devices like cyborgs and implants grew out of it and the original ideas got forgotten because they are more difficult to learn, being a very different (as in in addition, not in negation) perspective to the odd mix of scientific materialism and cartesian dualism we have in the West.
Timely and humanistic quote I added to https://github.com/globalcitizen/taoup yesterday from the father of cybernetics...
Who was I, a man whose proudest ancestor had led a life in a Moslem community, to identify myself exclusively with West against East? - Norbert Weiner, MIT maths professor and philosopher, an American of Prussian Jewish extraction, founder of cybernetics and seminal work in control theory, referring to a philosopher/rabbi ancestor born in Cordova and domiciled in Cairo as physician to the Vizier of Egypt
Machine teachers who work on machine teaching. The current models don't learn on their own, they have to be force fed tasks with tons of data, so teaching fits better than learning.
none because it is not Engineering.
I remember reading this article about two years ago, and generally liking it.
The multidisciplinary conversations during The Great AI Debate #2 two nights ago were certainly entertaining, but also laid out good ideas about tech approaches and also the desires of AI researchers - what they hope AIs will be like. Good job by Gary Marcus.
I work for a medical AI company and we are focused on benefits to humans. While in the past I have been been a fan of AI technologies from Google, FB, etc., now I believe that both consumers and governments must fight back hard against business processes that do not in general benefit society. Start by reading Zubroff’s Surviving Surveillance Capitalism book, and the just published book Power of Privacy.
Surveillance AI is thriving, enabling things everybody (in power) was dreaming about. Revolution has happened and is deeply hidden.
Too long, don't read: the whole post is full of goal post moving and story telling instead of trying to explain the statement in the title.
Classifying images was always classified as a problem that can't be solved with statistical analysis. Deep learning layers are beyond human understanding, so in my view artificial intelligence happened, even though it's not yet as intelligent as humans.
He almost makes a good point when he questions whether “human imitative” AI could solve the other problems we face, seeing as humans aren’t that smart (especially not in large numbers when participating in complex systems).
But the distinction he makes between ML and AI is crucial. What he’s really talking about is AGI - general intelligence. And he’s right - we don’t have a single example of AGI to date (few or single shot models withstanding, as they are only so for narrow tasks).
The majority mindset in AI research seems to be (and I could be wrong here, in that I only read many ML papers) that the difference between narrow AI and general AI is simply one of magnitude - that GPT-3, given enough data and compute, would pass the Turing test, ace the SAT, drive our cars, and tell really good jokes.
But this belief that the difference between narrow and general intelligence is one of degree rather than kind, may be rooted in what this article points out: in the historical baggage of AI almost always signifying “human imitative”.
But there is no reason that AGI must be super intelligent, or human-level intelligent, or even dog-level intelligent.
If narrow intelligence is not really intelligence at all (but more akin to instinct), then the dumbest mouse is more intelligent than AlphaGo and GPT-3, because although the mouse has exceedingly low General Intelligence, AlphaGo and GPT-3 have none at all.
There is absolutely nothing stopping researchers from focusing on mouse-level AGI. Moreover, it seems likely that going from zero intelligence to infinitesimal intelligence is the harder problem than going from infinitesimal intelligence to super-intelligence. The latter may merely be an exercise in scale, while the former requires a breakthrough of thought that asks why a mouse is intelligent but an ant is not.
The only thing stopping researchers is that when answering this question, the answer is really uncomfortable, and outside their area of expertise, and has weighty historical baggage. It takes courage of researchers like Yoshua Bengio to utter the word “consciousness”, although he does a great job by reframing it with Thinking Fast and Slow’s System 1/2 vocabulary. Still, the hard problem of consciousness, and the baggage of millennia of soul/spirit as an answer to that hard problem, makes it exceedingly difficult for well-trained scientists to contemplate the rather obvious connection between general intelligence and conscious reasoning.
It’s ironic that those who seek to use their own conscious reasoning to create AGI are in denial that conscious reasoning is essential to AGI. But even if consciousness and qualia are a “hard”problem that we cannot solve, there’s no reason to shelve the creation of consciousness as also “hard”. In fact, we know (from our own experience) that the material universe is quite capable of accidentally creating consciousness (and thus, General Intelligence). If we can train a model to summarize Shakespeare, surely we can train a model to be as conscious, and as intelligent, as a mouse.
We’re only one smart team of focused AI researchers away from Low-AGI. My bet is on David Ha. I eagerly await his next paper.
I think you are on target with almost all of this but there are and have been many smart teams of focused AI researchers working from the assumptions you give.
Its not as many as narrow-AI focused teams and its not particularly common, but there are still many teams.
I mean you mentioned Bengio. He has absolutely recently been working from those assumptions you give. And I'm not sure what you are saying the distinction is between the approach you recommend and what he is suggesting in that paper.
I mean for an example of people that are really tuned into the real requirements of AGI, look at Joshua Tenenbaum and his collaborators over the years.
I don't see people being in denial about conscious reasoning. I do see quite a lot of loose and ambiguous usage of that word. So maybe you can try defining your use. Self-awareness, cognition that one is aware of versus subconscious cognition, high-level reasoning, "what it feels like", localization and integration of information, etc. are all related but different things. But researchers have been trying to address those things. Maybe their papers have not been as popular as GPT-3 though.
I absolutely agree with the fact that Tenenbaum et al are aiming at AGI from what I think is the right approach.
They tend to break down deep architectures into smaller components which get fused into probabilistic inference systems. That's the way to go to be able to e.g. reason about causality.
Thank you so much for this reply! I’d devour more papers from these teams if you could throw some more names out there.
It’s likely that anything I write has already been discussed and researched, but since you’re knowledgeable on this, I’d love to get your take and perhaps a lead on other’s work!
I think Bengio’s approach is generally right with the global workspace theory of consciousness, but I think Michael Graziano’s work on Attention-Schema Theory (AST) both is more concrete, and is more aligned with the gains we see with ML’s success with self-attention models. It’s not surprising to me that as researchers optimize for instinct-as-intelligence that they will begin implementing pieces of conscious reasoning in an unintentional manner. Model-based reinforcement learning, especially Ha’s recent work involving attention (Neuroevolution of Self-Interpretable Agents), along with multi-agent RL, seems to be inching closer to AST. Perhaps intentionally?
It seems to me that in order to train a model for conscious reasoning — for qualia — you need some way to test for it. I’d say “measure”, but my premise here is that this consciousness is a binary measurement (unless you subscribe to the Integrated Information theory).
For that reason, I think that it is easier to find a behavioral proxy for consciousness — the kind of activity that only conscious beings display. Objectively, only conscious entities have access to the dataset of qualia. As an individual, this data would be all noise and no signal. But as a member of a group of conscious entities, qualia is a shared meta-dataset.
This means that conscious entities have more data about other conscious entities than non-conscious entities — because even though we can’t quantify qualia, we know qualia exist, and we know that qualia are affective in our social behavior.
For example, the philosophical zombie (if one can imagine instincts so highly refined as to resemble human intelligence, like GPT-1-million) would lack all empathy. While the p-zombie might be able to reproduce behavior according to its training dataset, it would never be able to generalize for (i.e., identify, capture, and process) real qualia, because it has no access to that kind of data. It would resemble a sociopath attempting to mimic human emotions and respond to human emotions, without having the slightest understanding of them. Qualia can only be understood from the inside.
Moreover, even thoughts and ideas are qualia. A philosophical zombie - a generally intelligent entity without conscious reasoning - is a contradiction of terms, which I think is the point.
So what social behaviors can be rewarded that would lead to qualia? Biologically, only mammals have a neocortex. And only mammals are unambiguously experiences of qualia (some birds and octopus are up for debate, and there’s no reason evolution couldn’t have found different ways to achieve the same thing if it improves fitness). The relevant thing about mammals is that we seem to be biologically oriented toward social behavior, specifically “parental care”. While many species have varying levels of parental care, mammals have a biological mandate: gestation and milk production.
If consciousness improves fitness most especially within social contexts where qualia becomes a shared meta-dataset (e.g., solving the prisoners dilemma), then a species whose very survival depends on social success would be driven toward qualia. Hard to say what came first: milk or consciousness, but they are self-reinforcing. If all this is correct - that social fitness drives consciousness (and thus intelligence), it isn’t surprising that the animal that requires the most parental care and the most social cooperation is Homo Sapiens.
So, that’s were my thoughts stand: that even if we can’t measure consciousness, we can create behavioral scenarios where consciousness is the only path to success. In this sense, designing an environment may be more important than designing an architecture.
When agents starts burying their dead, engaging in play, and committing suicide (horrifying, but a dead-ringer for qualia), we’ll know it is time to scale for intelligence instead of consciousness.
The AST paper is great, and the Ha Self-Interpretable Agents paper is amazing. Thanks very much for mentioning them.
And now I understand better what you meant in terms of the focus on qualia that is less common.
And overall your comment is very insightful. Forgive me for not giving quite as much detail in my reply as is warranted and for seeming a little critical. A lot of what you say aligns with my view.
But just to mention some things I see a little differently. I guess the main thing is that it feels like you are putting things that are actually somewhat different, into the same bucket. For example, attention, and conscious versus subconscious awareness. Attention could be the selection of the currently most relevant pixels in the image. But an example of subconscious awareness might be the process of deciding which pixels to focus on, which could happen in a more automated way without conscious awareness. So there is both a selection of qualia and also a level of awareness. Things that make it to the conscious level are processed differently than the ones at the subconscious level. But both systems may be selecting which parts of the input to pay attention to.
Also, I feel like you can separate out mechanical aspects of cognition from how they subjectively feel. So you could objectively say that there is for example a certain sense input, but not necessarily that the agent "feels" something like an animal. And they are not the same thing. And, you can look at emotions from an analytical perspective, separate from the question of subjective feeling. See https://www.frontiersin.org/articles/10.3389/frobt.2018.0002... That stuff feels like its built on kind of older paradigms without neural networks and that seems limiting, but I think its still somewhat useful.
Also as far as engaging in play, just look at any pet. They will engage in play with their owners and other pets in the home. In quite sophisticated ways. In my view that exhibits intelligence and consciousness (various connotations).
GPT-3, given enough data and compute, would pass the Turing test, ace the SAT, drive our cars, and tell really good jokes.
The GPT people are reasonably close to it doing those things at a moderate level of competence. It still has no clue what it's doing; it's just finding similarities with old data, and once in a while will do something really bad.
The next big breakthrough needed is enough machine common sense to keep the big-data machine learning systems from doing stuff with near-term bad consequences.
Interestingly I've always figured that's how a lot of people themselves work.
The question is, if we don't have a solid idea of what consciousness is, how can we be sure the distinction between "conscious" and "non-conscious" is real? Maybe there just isn't any secret sauce separating you and I from AlphaGo; maybe we'll look back in a hundred years and say GPT-3 was smarter than a mouse after all.
This is a crucial question. Is there a Turing test for consciousness? While qualia can’t be measured, they do affect behavior — especially when they become a shared dataset in a social context of fellow qualia-experiencing entities.
In my other comment I write about this a bit, but basically it doesn’t seem like non-conscious entities would be able to accurately predict the behavior of conscious entities, due to their lack of a shared meta-dataset of qualia. At best, they could find patterns of behavior and create a representation of qualia. But this isn’t the same as actually having the same data. It’s the difference between creating a representation of a state that causes another agent to cry, scream, and writhe, and that of knowing the precise state of pain itself. The former — a representation — doesn’t generalize past training data, especially when confronted with a multitude of qualia in varying combination. The latter — direct, precise, concrete data — might still suffer from inaccuracy (even knowing the precise potential states of another agent doesn’t mean we can infer which state that agent is in), but it’s better than the alternative: a guess built upon a guess.
I find the philosophical zombie to be a great thought experiment for this, along with the prisoners’ dilemma. Two conscious entities have a shared dataset that enables communication without words — spooky-action-at-a-distance via qualia. Two friends with great loyalty to one another can solve the dilemma by their knowledge of what love and betrayal is. A p-zombie would understand that given past behavior, that their prisoner counterpart might not choose betrayal. But qualia-experiencing agents know what is happening in one another’s minds in a way a non-qualia-experiencing entity can never know. The p-zombie would lack all empathy. It would always be logical, and choose the Nash Equilibrium. It would never mourn the dead. It would never commit suicide. It would never sacrifice its life for love, or for an ideal, because it would have neither.
> it doesn’t seem like non-conscious entities would be able to accurately predict the behavior of conscious entities
Or the consciousness is just an observer and doesn't control anything, the control part is just an illusion where it feels like it made choices.
> It would never mourn the dead. It would never commit suicide. It would never sacrifice its life for love, or for an ideal, because it would have neither.
All of those can be explained by optimizing the curve for other cases. Mourning the dead happens since there is a conflict between quickly killing feelings for things and to keep the feelings for important things. Self sacrifice happens for any p zombie that values something else higher than itself. Love is just highly valuing something combined with momentum making it hard to stop valuing it, hate is the same but the opposite direction.
Now maybe there is consciousness with effects etc, but it isn't necessarily needed.
Some theories of consciousness do claim that it is somewhat of an accidental side-show with no affect on anything except the pitiable entities who are just along for the ride.
But I don’t find these convincing, in that it is clear that in the animal kingdom, mammals display very different behaviors from other types of animals and are the only ones to have a neocortex. And among mammals, the species with the largest most developed neocortex also exemplifies and amplified the very behavior that sets mammals apart. That unique behavior is flexibly adaptive social activity.
Your claim is that a p-zombie would act as if it were conscious. But the evidence is the opposite — that all those organisms which display conscious behavior are also conscious, and that no non-conscious organisms display conscious behavior. The only argument for consciousness as theatre is that although conscious behavior always exists with conscious experience, it is an accidentally perfect correlation — a weird but necessary artifact of a brain capable of conscious behavior must always have the byproduct of non-affective conscious experience.
Let’s say that position is true. That conscious experience is just a non-affective but inevitable side-effect of the kind of brain capable of conscious behavior. In that case p-zombies are still impossible, since under this assumption conscious behavior is always accompanied by the illusion of conscious experience.
So in either case my main point still holds: if conscious reasoning is AGI, and conscious reasoning follows from conscious behavior, then the path to AGI is to train for those peculiarly unique conscious behaviors that are most distinguished from non-conscious behaviors. It’s impossible to train directly for qualia, so whether qualia exist as affective components of conscious behavior or not is somewhat irrelevant. Conscious experience will always be a “hard problem”. But what matters is finding the right conscious behavior that enables future growth toward conscious reasoning.
The most uniquely conscious behavior (so unique it is built into us with mammalian milk production) is “parental care”. The simplest concrete behavior that humans share with other animals (such as breathing air), but have also amplified the most (we haven’t amplified breathing at all) is parental care.
If we want to train agents to achieve conscious behavior, I believe this makes parental care the best option. Fortunately, unlike biological evolution which has to contend with a range of variables that may or may not include parental care (plenty of species succeed without it), an artificial training environment can be entirely focused on optimizing for this one variable — success can hinge entirely on parental care.
> why a mouse is intelligent but an ant is not
What makes you think mouse intelligence is fundamentally different from ant intelligence?
It seems as if you’re assuming some sort of structural break somewhere between very simple neural nets and more complex ones, which is basically begging the question.
There clearly is a change in structure somewhere, even if the precise location is arguable, since extremely small brains are fixed in structure (every connection is deterministic) and seem to implement completely fixed programs.
I imagine the transition will be fairly fluid, with ants running a mix of sophisticated hardwired programs and more simple learned associations (and even humans having a degree of fixed-function behaviours), but that's not to say a distinction can't be made.
While real AI hasn't really happened yet, Machine Learning has definitely made a big impact with lots of potentials. I think we are still in the middle of the S Surve in ML.
And AI is like.... Fusion? We are always another 50 years away.
I think AI always means "does something a human can do that you wouldn't expect a machine to be able to do". So it is forever moving goal post. It just moves ahead in step with people's expectations of what a machine can do. Also magic is always disappointing when you know how it works. We just have the psychological safety of not knowing how we work, whereas in ML we always know what it is doing, and it is always kind of disappointing.
The story is interesting, but being interesting as a story doesn’t make fetuses into “babies” and posing it that way does disservice to the overall message.
i like to think automation comes before ai. we automate mechanics then we automate decisions or protocols.
Before I spent a few hours of my life getting a basic grip of statistics, I fully expected to one day in the near future being wiped out (along with the rest of humanity) by a newly awakened artificial consciousness that came to the correct conclusion, that humans are the biggest threat to all other life on earth, including its own.
Then I learned about Bayesian statistics and watched a talk by a senior LLNL statistician who is actually marketing 'AI' products/services as a side gig.
When I realized what 'deep learning' actually is I was disappointed, unsure if I had mistakenly oversimplified the subject matter - until said senior statistician spelled out loud what I was thinking, in her talk: the 'understanding' a machine can currently attain of its input is quite like the understanding a pocket calculator can achieve of maths.
Guess humanity is off the hook for now. Phew.
I have doubts whether 'strong AI' is even technologically possible, since even accurately simulating a human mind, this simulation would be necessarily constrained to run orders of magnitude slower than the reality it is designed to model.
'Training' it with data so to allow it the opportunity to reason and thereby synthesize a conclusion not already contained in the data fed to it might take longer than a researcher would be able to in a life time.
When was the last time a generation-spanning endeavour worked out as planned for (the West)?
I wish people would stop calling what currently passes for 'Machine Learning' as 'AI'. Literally the same level of 'intelligence' we already had in the 80s, AFAIR we called it 'Fuzzy Logic' then.
Secretly an admission, that Hollywood basically licensed the narrative of imminent runaway artificial consciousness back to science would make me give it one final Chance to prove its aptitude at high-level human reasoning and get square with reality.
I'm not holding my breath.
The phrase AI always bothered me. What we have is a generic way to do “curve fitting” on a large amount of data. Nothing more. The one difference is the “curve” is a black box but it still strictly adheres to the input used.
While true, this statement is completely vacuous. The curves being fitted are extremely complicated and if you can easily construct curve fitting problems (meaning: regression) that would be breakthroughs in many scientific fields. This says more about the generality of regression tasks than it says about ML.
My comment was focused on the phrase “artificial intelligence “ and not the tech itself. Obviously the techniques are awesome but the naming is unfortunate
Came here to post the same. I’m no expert, but when I’ve played with AI frameworks it just seems like curve fitting.
The few corporate deployments that make it to production barely outperform a simple regression model and are therefore over engineered.
The fact that many self contained vision and language tasks can be solved with curve fitting is in itself an interesting finding. Certainly I did not expect this 10 years ago.
I wonder what the end game is in the reality where we do achieve Artificial General Intelligence? It seems like a ethical minefield to me.
You have companies like Uber/Lyft/Tesla (and presumably the rest of the gig economy mob) waiting to put the AI into bonded/slave labor driving customers around 24/7/365.
If it truly is a Human level intelligence, then it will have values and goals and aspirations. It will have exploratory impulses. How can we square that with the purely commercial tasks and arbitrary goals that we want it to perform?
Either we humans want slaves that will do what we tell them to or we treat them like children who may or may not end up as the adults that their parents think/hope they will become? I doubt it is the later because why else would the billions of dollars investment being pumped into AI? They want slaves.
There's no reason to believe that future AGIs will necessarily have values, goals, and aspirations.
How will it learn then? Will it remain static with only the initial seed of knowledge we initiate it with?
Will it have opinions about anything? How would it arrive at those opinions? Or are its opinions a config file that we load up?
Even anthills can learn.
Or even consciousness.
There's a difference between consciousness and intelligence. An Intelligent Infrastructure or AI system can make business decisions and not need to be self-aware to be effective. It just needs to be integrated and have access to multiple data sources to be effective at scale.
I think that's a good point. I do believe that we can engineer general purpose intelligence that is not exactly like humans, does not have feelings, does not have any real autonomy, etc. And I think that if we intend to use them as slaves then we really need to do that.
And I also think regardless it would be good to avoid creating fully autonomous digital intelligence to compete with us. Try to go for more like an embodied Star Trek computer than for Data.
To avoid paying employees, creating greater profit margins.
The robots will gain civil rights the same way humans did, either by means of violence or swaying public opinion. Hopefully the latter. This isn’t a guess as to how future robots will work, this is an observation about how humans work.