Mark Humphrys - Publications

Publications

The World-Wide-Mind

A novel application of Web Services in Computer Science education, Ciarán O'Leary, Mark Humphrys and Ray Walshe, in The International Conference on "Computer as a tool" (EUROCON 2005), Belgrade, Serbia & Montenegro, November, 2005.

Constructing Complex Brains: Building minds using sub-minds from biotechnology authors, Walshe, Ray; Humphrys, Mark and O'Leary, Ciarán (2004), 1st IFIP Conference on Artificial Intelligence Applications and Innovations (AIAI-04), Toulouse, France, August 22-27, 2004 (see also here and here).

Constructing an animat mind using 505 sub-minds from 234 different authors, O'Leary, Ciarán; Humphrys, Mark and Walshe, Ray (2004), poster in Stefan Schaal et al., eds., From Animals To Animats 8: Proceedings of the 8th International Conference on the Simulation of Adaptive Behavior (SAB-04), July 2004, Los Angeles, CA, pp.39-48. You can order the proceedings from MIT Press. Abstract - The World-Wide-Mind (WWM) is a scheme for putting minds (by which we mean the software to drive a real or virtual agent) online for remote re-use by third parties. Although the animat community favours the extension of existing agent minds in an ongoing process, there is currently no easy way to re-use minds built by other authors. If re-use became the norm, it is suggested that larger, more diverse and more complex minds could be built. This paper describes an ongoing project that uses WWM technology to develop a novel set of minds that incorporate a range of algorithms and techniques. We demonstrate that it is possible to evolve superior minds by the artificial selection and combination of existing online minds. The project that is described here involved 234 authors who developed 505 different minds for the virtual animal in a re-implementation of Tyrrell's Simulated Environment (Tyrrell, 1993). These minds, which were developed by two classes of undergraduate Computer Science students, were subsequently selected based on their performance in the virtual world, and integrated into larger minds. At the time of writing the most successful mind developed used as components the two sub-minds that most successfully satisfied the animal's two main goals. Since a huge number of combinations of minds is possible, it is important that the work is distributed among a community of researchers. The architecture of the World-Wide-Mind makes this possible.

O'Leary, Ciarán and Humphrys, Mark (2003), Building a hybrid Society of Mind using components from ten different authors, poster in Wolfgang Banzhaf et al., eds., Advances in Artificial Life: Proceedings of the 7th European Conference on Artificial Life (ECAL-03), Dortmund, Germany, September 14-17, 2003, published by Springer-Verlag, LNAI 2801, pp.839-46. © Springer-Verlag.

• Springer-Verlag
  • LNCS/LNAI Series (also here)
    • LNAI 2801

Humphrys, Mark and O'Leary, Ciarán (2002), Constructing complex minds through multiple authors, in Bridget Hallam et al., eds., From Animals To Animats 7: Proceedings of the 7th International Conference on the Simulation of Adaptive Behavior (SAB-02), August 2002, Edinburgh, Scotland, pp.3-12. You can order the proceedings from MIT Press (or via here). Abstract - The World-Wide-Mind (WWM) was introduced in [Humphrys, 2001]. For a short introduction see [Humphrys, 2001a]. Briefly, this is a scheme for putting animat "minds" online (as WWM "servers") so that large complex minds may be constructed from many remote components. The aim is to address the scaling up of animat research, or how to construct minds more complex than could be written by one author (or one research group). The first part of this paper describes how a number of existing animat architectures could be implemented as WWM servers. Any unified mind can easily map to a single WWM server. So most of the discussion here is on action selection (or behavior or goal selection), where each module could be a different WWM server (written by a different author). The second part of this paper describes the first implementation of WWM servers and clients, and explains in particular how to write a WWM server. Most animats researchers are programmers but not network programmers. Almost all protocols for remote services (CORBA, SOAP, etc.) assume the programmer is a networks specialist. This paper rejects these solutions, and shows how any animats researcher can put their animat "mind" or "world" online as a server by simply converting it into a command-line program that reads standard input and writes to standard output. Follow online references (and here).

O'Leary, Ciarán and Humphrys, Mark (2002), Lowering the entry level: Lessons from the Web and the Semantic Web for the World-Wide-Mind, poster presented at the 1st International Semantic Web Conference (ISWC-02), June 2002, Sardinia, Italy.

• See 1 page extended abstract. This is in Posters book (or via here), p.36.

O'Connor, Dave and Humphrys, Mark (2002), The Implementation of a Distributed Hierarchical Mind on the Internet using the World-Wide-Mind, Dublin City University, School of Computing, Technical Report no. CA-0302.

• O'Connor, Humphrys and Walshe, First Implementation of the World-Wide-Mind.
• O'Connor, AIML - A Markup for Communication on the World-Wide-Mind.
• O'Connor, Subclassing of World State Data in Grid-like Worlds.
• O'Connor, Representing the Action   a   in Grid-like Worlds.

Humphrys, Mark (2001), Distributing a Mind on the Internet: The World-Wide-Mind, in Jozef Kelemen and Petr Sosik, eds., Advances in Artificial Life: Proceedings of the 6th European Conference on Artificial Life (ECAL-01), Prague, Czech Republic, published by Springer-Verlag, LNAI 2159, pp.669-80. © Springer-Verlag. Abstract - It is proposed that researchers in AI and ALife construct their agent minds and agent worlds as servers on the Internet. Under this scheme, not only will 3rd parties be able to re-use agent worlds in their own projects (a long-standing aim of other schemes), but 3rd parties will be able to re-use agent minds as components in larger, multiple-mind, cognitive systems. Under this scheme, any 3rd party user on the Internet may select multiple minds from different remote "mind servers", select a remote "Action Selection server" to resolve the conflicts between them, and run the resulting "society of mind" in the world provided on another "world server". Re-use is done not by installing the software, but rather by using a remote service. Hence the term, the "World-Wide-Mind" (WWM), referring to the fact that the mind may be physically distributed across the world. This model addresses the possibility that the AI project may be too big for any single laboratory to complete, so it will be necessary both to decentralise the work and to allow a massive and ongoing experiment with different schemes of decentralisation. We expect that researchers will not agree on how to divide up the AI work, so components will overlap and be duplicated and we need multiple-conflicting-minds models [Humphrys, 1997]. We define the set of queries and responses that the servers should implement. Initially we consider schemes of low-bandwidth communication, e.g. schemes using numeric weights to resolve competition. This protocol may initially be more suitable to sub-symbolic AI. The first prototype implementation is described in [Walshe and Humphrys, 2001]. It may be premature in some areas of AI to formulate a "mind network protocol", but in the sub-symbolic domain it could be attempted now. Springer-Verlag LNCS/LNAI Series (also here) LNAI 2159 (also here) this paper (or via here)

Walshe, Ray and Humphrys, Mark (2001), First Implementation of the World-Wide-Mind (also here), poster in Jozef Kelemen and Petr Sosik, eds., Advances in Artificial Life: Proceedings of the 6th European Conference on Artificial Life (ECAL-01), Prague, Czech Republic, published by Springer-Verlag, LNAI 2159, pp.714-8. © Springer-Verlag.

Humphrys, Mark (2001), The World-Wide-Mind: Draft Proposal (and here), Dublin City University, School of Computing, Technical Report no. CA-0301. Length - 54 pages. Abstract - In the first part of this paper, a change in methodology for the future of AI and Adaptive Behavior research is proposed. It is proposed that researchers construct their agent minds and their agent worlds as servers on the Internet. 3rd parties will use these servers as components in larger systems. In this scheme, any user on the Internet will be able to (a) select multiple minds from different remote "mind servers", (b) select a remote "Action Selection server" to resolve the (inevitable) conflicts between these minds, and (c) run the resulting constructed "society of mind" in the world provided on another "world server". All this without necessarily having to consult with the server authors. This constructed society may now also be presented as just another primitive mind server, ready for reuse by others as a component in a larger system. From the current situation of isolated experiments we will move to a situation where not only can researchers use each other's agent worlds, but they can also use each other's agent minds as components in larger systems. Servers may call other servers, and it is expected that 3rd parties will continuously write wrappers and filters for existing mind servers, overriding and modifying their default behaviour (to produce new, co-existing mind servers). None of this necessarily means that the mind being used ever leaves its server (or that its insides are even made public). Hence the term, the "World-Wide-Mind" (WWM), referring to the fact that the mind may be physically distributed across the world, with parts of the mind at different remote servers. Part of the motivation for the WWM is that if the AI project is to be successful, it may be too big for any single laboratory to complete. So it will be necessary both to decentralise the work and to allow a massive and ongoing experiment with different combinations of components (so that we are not locked into any particular layout of decentralisation). Central to the WWM scheme is the expectation that researchers will not agree on how to divide up the AI work, and so components will overlap and be duplicated. Previous work by this author [Humphrys, 1997] introduced models of mind where competition took place between extremely incompatible components, and where the mind could survive communications failure with or even complete loss of a number of such components. The WWM idea grew out of this work, and this paper shows how these previous models are the type of models we need in the WWM. In the second part of this paper, we move towards an implementation of the WWM by trying to define the set of queries and responses that the servers should implement. Clients (including other servers) may then implement any general-purpose algorithm to drive the servers through repeated use of these queries. In our initial implementation, we consider schemes of very low-bandwidth communication. For instance, schemes where the competition among multiple minds is resolved across the network using numeric weights, rather than by explicit reasoning and negotiation. It is possible that this low-bandwidth protocol may be more suitable to sub-symbolic AI than to other branches of AI, and that other protocols may be needed for other branches of AI. It is suggested that it may be premature in some areas of AI to attempt to formulate a "mind network protocol", but that in the sub-symbolic domain it could at least be attempted now. Whether the protocol presented here is adopted or not, the first part of this paper (the need for a protocol) stands on its own. Finally, we suggest a lowest-common-denominator approach to actually implementing these queries, so that current AI researchers have to learn almost nothing in order to put their servers online. As the lowest-common-denominator approach we suggest the transmission across ordinary CGI of queries and responses written in XML. Keywords - World-Wide-Mind, WWM, Network Minds, Distributed Models of Mind, Society of Mind, Distributed AI, autonomous agent architectures, Action Selection, Internet, client-server, HTTP, CGI, XML, AIML. See Draft list of WWM server queries. See Draft ideas for AIML. Follow online references.

Action Selection / sub-symbolic Society of Mind

Humphrys, Mark (1997), Action Selection methods using Reinforcement Learning (also here and here and here and here), PhD thesis (expanded version, produced as Technical Report no.426), University of Cambridge, Computer Laboratory. You can read this in the department library or you can order a bound copy. Length - 195 pages. Abstract - The Action Selection problem is the problem of run-time choice between conflicting and heterogenous goals, a central problem in the simulation of whole creatures (as opposed to the solution of isolated uninterrupted tasks). This thesis argues that Reinforcement Learning has been overlooked in the solution of the Action Selection problem. Considering a decentralised model of mind, with internal tension and competition between selfish behaviors, this thesis introduces an algorithm called "W-learning", whereby different parts of the mind modify their behavior based on whether or not they are succeeding in getting the body to execute their actions. This thesis sets W-learning in context among the different ways of exploiting Reinforcement Learning numbers for the purposes of Action Selection. It is a "Minimize the Worst Unhappiness" strategy. The different methods are tested and their strengths and weaknesses analysed in an artificial world. See accompanying Movie demo - The House Robot problem. See accompanying Movie demo - W-learning in the Ant World problem. PhD Supervisor: John Daugman, Internal examiner: Richard Prager, External examiner: Tony Prescott. Follow online references.

Humphrys, Mark (1996), Action Selection methods using Reinforcement Learning, PhD thesis, University of Cambridge, Computer Laboratory. You can read this as PhD 20843 in the Manuscripts Room of Cambridge University Library. This is just here for completeness. Please see instead the expanded technical report above.

Humphrys, Mark (1996), Action Selection methods using Reinforcement Learning (also here), in Pattie Maes et al., eds., From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior (SAB-96), Massachusetts, USA, pp.135-44. You can order the proceedings from MIT Press/Bradford Books. Length - 10 pages. Abstract - Action Selection schemes, when translated into precise algorithms, typically involve considerable design effort and tuning of parameters. Little work has been done on solving the problem using learning. This paper compares eight different methods of solving the action selection problem using Reinforcement Learning (learning from rewards). The methods range from centralised and cooperative to decentralised and selfish. They are tested in an artificial world and their performance, memory requirements and reactiveness are compared. Finally, the possibility of more exotic, ecosystem-like decentralised models are considered. See accompanying Movie demo - The House Robot problem. See accompanying Movie demo - W-learning in the Ant World problem.

Humphrys, Mark (1996), Action Selection in a hypothetical house robot: Using those RL numbers (also here), in Peter G.Anderson and Kevin Warwick, eds., Proceedings of the First International ICSC Symposia on Intelligent Industrial Automation (IIA-96) and Soft Computing (SOCO-96), Reading, England, pp. B 216-22. You can order the proceedings from ICSC Academic Press.

Humphrys, Mark (1995), Towards self-organising Action Selection (also here), in Sam Steel, ed., Papers of the 14th Workshop of the UK Planning and Scheduling Special Interest Group (SIGPLAN 14) (or here), University of Essex, Department of Computer Science, Technical Report no.255 (try here and here).

Humphrys, Mark (1995), W-learning: A simple RL-based Society of Mind (also here), poster presented at the 3rd European Conference on Artificial Life (ECAL-95), Granada, Spain. Title page is in Posters book, p.30.

Humphrys, Mark (1995), W-learning: Competition among selfish Q-learners (also here and here), Technical Report no.362, University of Cambridge, Computer Laboratory. You can read this in the department library or you can order a bound copy. Length - 30 pages. Abstract - W-learning is a self-organising action-selection scheme for systems with multiple parallel goals, such as autonomous mobile robots. It uses ideas drawn from the subsumption architecture for mobile robots (Brooks), implementing them with the Q-learning algorithm from reinforcement learning (Watkins). Brooks explores the idea of multiple sensing-and-acting agents within a single robot, more than one of which is capable of controlling the robot on its own if allowed. I introduce a model where the agents are not only autonomous, but are in fact engaged in direct competition with each other for control of the robot. Interesting robots are ones where no agent achieves total victory, but rather the state-space is fragmented among different agents. Having the agents operate by Q-learning proves to be a way to implement this, leading to a local, incremental algorithm (W-learning) to resolve competition. I present a sketch proof that this algorithm converges when the world is a discrete, finite Markov decision process. For each state, competition is resolved with the most likely winner of the state being the agent that is most likely to suffer the most if it does not win. In this way, W-learning can be viewed as "fair" resolution of competition. In the empirical section, I show how W-learning may be used to define spaces of agent-collections whose action selection is learnt rather than hand-designed. This is the kind of solution-space that may be searched with a genetic algorithm. See accompanying Movie demo - W-learning in the Ant World problem.

The Turing Test

Humphrys, Mark (2008), "How my program passed the Turing Test", Ch.15 of Parsing the Turing Test: Philosophical and Methodological Issues in the Quest for the Thinking Computer, Robert Epstein, Gary Roberts and Grace Beber (eds.), Springer, 2008, pp.237-260. Abstract - In 1989, the author put an Eliza-like chatbot on the Internet. The conversations this program had can be seen - depending on how one defines the rules (and how seriously one takes the idea of the test itself) - as a passing of the Turing Test. This is the first time this event has been properly written up. This chatbot succeeded due to profanity, relentless aggression, prurient queries about the user, and implying that they were a liar when they responsed. The element of surprise was also crucial. Most chatbots exist in an environment where people expect to find some bots among the humans. Not this one. What was also novel was the online element. This was certainly one of the first AI programs online. It seems to have been the first (a) AI real-time chat program, which (b) had the element of surprise, and (c) was on the Internet. We conclude with some speculation that the future of all of AI is on the Internet, and a description of the "World-Wide-Mind" project that aims to bring this about. Keywords - chatbot, Turing Test, Eliza, Internet, chat, BITNET, CHATDISC. Written 2002. See accompanying web page - "How my program passed the Turing Test"

AI in general

• Apparently there was a RealVideo stream here: but I never saw it.

Humphrys, Mark (1997), AI is possible .. but AI won't happen: The future of Artificial Intelligence (also here and formerly here), talk given at the "Next Generation" symposium, the "Science and the Human Dimension" series, Jesus College, Cambridge. The proceedings were formerly on the New Scientist web site. The talk has also been republished in Neo magazine, June 1998.

• See accompanying web page - The Philosophy and Future of AI

Computers and Internet

I am interested in why people keep trying to make the Internet harder to use than it is. The Internet is full of incomers who don't seem to understand the medium - who think it is like TV and sites must grab attention - who think it is like CD-ROM's and it is all about multimedia - who cannot or refuse to recognise how slow it is. And who do not know what a hyperlink is, and certainly cannot imagine that anyone might ever want to link to them:

• See follow-up to this article.

Object-Oriented Programming

I am now experimenting in AI with what in the long run may be a more powerful method of re-use: leave the function at the remote server.

• From the document source:

  .* Security classification:  unclassified                         *
  

  and also in the document source:

  Feel free to distribute this report 
  (within or outside of IBM - it is unclassified).
  

Hypertext and Genealogy

Humphrys, Mark (2000),   "Hypertext Indented Narrative" pedigree format: Adapting the Burke's Peerage format for the Web, or: How to draw indefinitely large family trees by hand,   Computers in Genealogy,   7(1):26-46, Mar 2000, published by the Society of Genealogists, London. Abstract - This paper discusses how to add hypertext to the "Indented Narrative" pedigree format (which I refer to as "IN" format, and which is exemplified by the format used in Burke's Peerage). The result is a free-form, hand-drawable, hyperlinked data format which I call "Hypertext Indented Narrative", or "HIN" format. This paper analyses the merits of HIN and argues for it as a real alternative to the rigid and constraining databases that have dominated genealogy since the advent of the computer (but which in many ways represent a philosophy from the era of the PC, i.e. before the advent of the Web). It is argued that the main advantage of the IN format was its variable resolution - an advantage that is lost and unappreciated in the modern databases. It is argued that there were some basic problems with the IN format when implemented on paper, that prevented it reaching its full potential. These problems can be overcome with the simple and controlled use of hypertext. The result is a format that allows one for the first time to properly draw and maintain networks of arbitrarily-complex interconnected pedigrees entirely by hand. This is a real alternative to the usual practice of wrestling one's information into the limited pre-defined fields of some rigid database. The (much-underplayed) disadvantages of databases and the possible advantages of free-form formats are discussed. Finally, HIN does not have to be an alternative to databases - It is also argued that HIN could be automatically generated as one of the standard output formats of a database. Keywords - Hypertext, Genealogy, Pedigree formats, Indented Narrative pedigree format, Burke's Peerage. See accompanying web page - FAQ - Why do I use a hypertext narrative to draw pedigrees Follow online references.

Links (scientific paper archives and search engines)

• DCU library
  • web catalog
  • Online databases that DCU is subscibed to.
  • ACM
  • Springer
  • ISI
  • Web of Science
  • LNCS

• Google Scholar

• CiteSeer
  • Computer Science
  • Machine Learning
  • Reinforcement Learning

• CogPrints
• arxiv.org

• DBLP Computer Science Bibliography
  • journals
  • conferences
  • AI
  • Machine Learning

• Computer Science Bibliographies
  • AI

Return to Home page.