Neural Networks and Learning

• Neural networks for combinatorial optimization, back-propagation, and protein folding share a common problem which tends to increase rapidly with size. Describe the generic problem, and approaches to circumvent it in one of these cases.
• Two classic algorithms for training an individual neuron are the "adaline" and the "perceptron" learning algorithm. Describe the differences between the two methods and discuss the practical consequences of these differences.
• Most real-life variational problems have a non-convex energy landscape. Describe stochastic search methods (such as simulated annealing, genetic algorithms) for locating the global minimum.
  stochastic.pdf
• Discuss the role of the non-linearity in two-layer networks (i.e. networks with a hidden layer). Compare thresholding versus soft sigmoidal functions and bump functions in terms of network function and learning capability.
• A person rolls two fair dice and records the total number of points. You can ask a sequence of yes/no questions to find out this number. The answer to a question may affect your choice of the following questions. Devise and justify a strategy that achieves the minimum possible average number of questions.
• What is the Central Limit Theorem?
• Discuss the concept of Shannon information as applied to the nervous system.
• What is Principal Component Analysis and how is it relevant within the context of neural networks?

Return to regular view	Text-only	Updated: 11/16/2009