"While we will emphasize formal tools throughout this book, we do not imply that formal tools are a panacea or that formal approaches are superior to informal ones in an absolute sense. (…) it’s very easy for formal approaches to get out of control, to the point where they are overly obscure, or require too much mathematical machinery to be of any practical use on a day-to-day basis."
"It should be noted that universal approximation is not a rare property. Many other systems have similar capabilities: polynomials, trigonometric polynomials (e.g., Fourier series), kernel regression systems, wavelets, and so on. By itself, this property does not make neural networks special. The results are important because they show that neural networks are powerful enough to approximate most functions that people find interesting. The lack of a universal approximation capability, on the other hand, would be bad news; neural networks would then be too weak for many problems and therefore much less appealing."
Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, by Russell D. Reed & Robert J. Marks II, Chapter 4.
"The term ‘neural network’ has its origins in attempts to find mathematical representations of information processing in biological systems (McCulloch and Pitts, 1943; Widrow and Hoff, 1960; Rosenblatt, 1962; Rumelhart et al., 1986). Indeed, it has been used very broadly to cover a wide range of different models, many of which have been the subject of exaggerated claims regarding their biological plausibility. From the perspective of practical applications of pattern recognition, however, biological realism would impose entirely unnecessary constraints."
"We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves."
"In fact, the perceived problems stem from the fact that real-name verifiability on the internet today is still not strong enough - the current “gold standard” of real names on the internet, Facebook, still allows you to create an entire false identity fairly easily - and anonymity technology is also not strong enough: unless you are very, very good with computers and cryptography, your identity can be uncovered by a determined (i.e. powerful) adversary even when you imagine you are being “anonymous” on the internet."
"Stop. Take a deep breath. Think about all that you have just discovered (or if you have already discovered it all, reflect on how awesome it is). By page 45 of SICP, you will have:
Translated recursive definitions into iterative processes;
Examined the differences in the space requirements of various processes;
Used tail-recursion to implement efficient iterative procedures;
Seen that iteration can be thought of as a special case of recursion;
Realized that iterative processes can be restarted easily by capturing and restoring their state variables;
Given thought to complexity, optimization, and state transition theory;
Been encouraged to explore creative solutions to challenging problems;
Learned the basic syntax and semantics of Scheme;
Observed a quasi-religious experience."
"The same psychological developments that help the popularity of videogames wind up hurting the cause of education since our society has decided, for better or worse, that self-confidence is too important to threaten."
"Some attempts have been made to find out the value that people place on their own lives. One common “currency” used in medical and safety analysis is the micromort, a one in a million chance of death. If you ask people how much they would pay to avoid a risk - for example, to avoid playing Russian roulette with a million-barreled revolver - they will respond with very large numbers, perhaps tens of thousands of dollars, but their actual behavior reflects a much lower monetary value for a micromort. For example, driving in a car for 230 miles incurs a risk of one micromort: over the life of your car - say, 92,000 miles - that’s 400 micromorts. People appear to be willing to pay about $10,000 (at 2009 prices) more for a safer car that halves the risk of death, or about $50 per micromort. A number of studies have confirmed a figure in this range across many individuals and risk types. Of course, this argument holds only for small risks. Most people won’t agree to kill themselves for $50 million."
"In his book Lectures on the Theory of Functions, J. E. Littlewood explained that there are three principles that lie behind work in real analysis:
1. Every measurable set is almost a finite union of open intervals.
2. Every measurable function is almost a continuous function.
3. Every convergent sequence of measurable functions is almost uniformly convergent."
"The relation between logic programming and Prolog is reminiscent of the relation between the lambda-calculus and Lisp. Both are concrete realizations of abstract computation models. Logic programs that execute with Prolog’s execution mechanism are referred to as pure Prolog. Pure Prolog is an approximate realization of the logic programming computation model on a sequential machine. It is certainly not the only possible realization. However, it is a realization with excellent practical choices, which balance preserving the properties of the abstract model with catering for efficient implementation."