Image Aspects (Bild-Aspekte) SS'24
Vorlesung

.

Notes:

• This lecture is conducted exclusively in presence.

• This lecture does not use studIP. Please register to attend the lecture using the registration form on our website (https://graphics.tu-bs.de/teaching/students). This way we can notify you of any current changes. This registration is informal and does not replace the official examination registration at the respective examination office when required.

• If you encounter any problems, please send an email to ba@cg.cs.tu-bs.de.

• The lecture will be conducted in English.

• On this week (18.05.2022) there will be no lecture!
• Please register to attend the lecture using the registration form on our website (https://graphics.tu-bs.de/teaching/students). This way we can notify you of any current changes. This registration is informal and does not replace the official examination registration at the respective examination office!
  If you encounter any problems, please send an email to ba@cg.cs.tu-bs.de.
  

Presentation:

• Allocation of presentation topics : first come, first serve
  The topics are now available, please send an e-Mail with the top 3 desired presentation topics (indicated by the code at the end of each paper) and 3 preferred presentation dates to: ba@cg.cs.tu-bs.de
• The topics and the available dates will be announced in this webpage.
• (mails received before this date will not be considered!).
  
• Starting of topic and presentation date registration: 25.04
• Deadline for topic and presentation date registration: 07.05

• Presentation dates:  5.6, 12.06, 19.06, 26, 06, 03.07. (The 10.07 is reserved).
• You can find the ICG slides template here. Its use is not mandatory.
• Please not that each presentation session consists (in principle) of 5 papers and the times have been calculated accordingly, i.e., for an optimal of 14 minutes talk plus 3 minutes questions. Please adjust to those times as any delay will result in running over the time of your colleagues or into a longer session.
• You re welcomed to use any software to prepare the presentation. If you want to use the lecturer's laptop the presentation must be exported to powerpoint, keynote, or pdf. In that case you need to send your presentation to ba@cg.cs.tu-bs.de not later than 9am on the day scheduled for your preentation.

• We will resume the lecture on the 22.06 with the first round of presentations.

Description:

What are pictures? How are they created? What distinguishes pictures? How do they impact us? The lecture explores the nature of images from the perspective of mathematics, physics, computer science, psychology, neuroscience, and art.

The course is explicitly aimed at students of ALL disciplines and is also offered as a key qualification in the lecture pool "interdisciplinary qualification" ("überfachliche Qualifikation").

Time and place:

• Every Wednesday from 15:00 to 16:30
• Room: IZ 161
• Lecture period: 02.04.2024 to 13.07.2024
• Start: 10.04.2024

Contents:

• Physical foundations of image formation
• Statistical and other properties of natural images
• Physiology of visual perception - Biological evolution of the human visual system
• Optical illusions and what they are good for
• Relationship between images and visual information
• Visual arts as experimental neuroscience
• How many pictures are there?
• Which is the oldest picture in the world?
• How do optical illusions work?
• Impossible shapes
• What makes faces so interesting?
• How does make-up work?
• Is attractiveness hereditary?
• Pictures on drugs
• Hidden image content
• Why do we hang impressionist paintings on our walls?
• What's behind modern art?

Formalities:

• Duration: 2 SWS
• Studienleistung - Non graded
• Examination modality: Presentation
• Attendance of  at least 50% of the course is necessary to successfully complete the module.

In order to obtain the course credits, each participant must prepare and give a short scientific presentation of around 12-14 minutes in length on one of the scholarly articles that will be listed below under "topics". Please not that each presentation session consists (in principle) of 5 papers and the times have been calculated accordingly, i.e., for an optimal of 14 minutes talk plus 3 minutes questions. Please adjust to those times as any delay will result in running over the time of your colleagues or into a longer session.  Topics will be assigned in late April / early May. Own suggestions for topics are also welcome! (please send the corresponding article in electronic form to the contact mail ba@cg.cs.tu-bs.de).

When the registration opens you will be asked to list three topic preferences.

Please list three topic preferences

Prerequisites:

• Interest in images, photography, art, vision, and visual perception.

Topics:

To be announced

All links are accessible from within the TU domain and the pdf files are downloadable. If there are any problems please report them to ba@cg.cs.tu-bs.de.

Images and Nature

• S. Nayar et al."Fast Separation of Direct and Global Components of a Scene using High Frequency Illumination", 2006 (Nat1)
  Ingeniously elegant method to record how objects would look without internal scattering.
  
  
• D. Weiskopf, D. Kobras, and H. Ruder "Real-World Relativity: Image-Based Special Relativistic Visualization", 2006 (Nat2)
  
  
• D. Weiskopf, U. Kraus, and H. Ruder "Searchlight and Doppler Effects in the Visualization of Special Relativity: A Corrected Derivation of the Transformation of Radiance", 1999 (Nat3)

Images and Numbers

• D. Corney, R. Lotto, "What are Lightness Illusions and why do we see them?", 2007 (N1)
  Neural network that "sees" the same optical illusions as we do.    
  

• E. Watanabe et al., "Illusory Motion Reproduced by Deep Neural Networks Trained for Prediction", 2017(N2)
  Deep Neural Net that perceives the famous "Rotating Snakes" illusion as we do.              
  

• B. Olshausen and D. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images", 1996 (N3)
  Computer experiment that shows how the functioning of certain neurons in our visual cortex can be explained.

• Wu et al."Eulerian Video Magnification for Revealing Subtle Changes in the World", 2012 (N4)
  Visualization of minimal local color and motion changes in video recordings based on frequency analysis.
  
  
• E. Reinhard et al., "Second order image statistics in computer graphics", 2004 (N5)
  Elegant application of natural image statistics to upscale pictures.
  
  
• C. Wang, N. Komodakisa, and Nikos Paragios, "Markov Random Field Modeling, Inference & Learning in Computer Vision & Image Understanding: A Survey", 2013 (N6)
  
  
• K. Nazeri et al., "Edge-Informed Single Image Super-Resolution", 2019 (N7)
  
  
• Meng et al. "SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations", 2022 (N8)
  
  
• G. Carlsson et al. "On the Local Behavior of Spaces of Natural Images", 2008 (N9)
  
  
• R. Ghrist, "Barcodes: the Persistent Topology of Data", 2008 (N10)
  
  
• Lee et al., "The Nonlinear Statistics of High-Contrast Patches in Natural Images", 2003 (N11)     
  
  

Images and Vision

• H. Kolb, "How the retina works", 2004 (V1)
  Excellent article about the structure and function of our retina.       
  
  
• E. Land, "The Retinex theory of color vision", 1977 (V3)
  Retinex theory: Computer model for modeling human color perception based on psychological experiments. See also:
  •  J. McCann, "Lessons learned from Mondrians applied to real images and color gamuts", 1999 (V3.1)
  • B. Funt et al., "Retinex in MATLAB", 2004 (V3.2)

• S. Magnusson et al., "Filling-in of the foveal blue scotoma", 2001 (V4)
  Why can we see the color blue even though we have no receptors for this color in the central fovea?        
  
  
• L. Riggs et al., "The Disappearance of Steadily Fixated Visual Test Objects", 1953 (V5)
  Classic experiment that shows that the image on the retina must move constantly, otherwise we would be blind!
  
  
• S. Shevell et al., "Color in complex scenes", 2008 (V6)
  Overall view of the mutual influence of colors in our perception.
  
  
• P. Bressan, "Explaining lightness illusions", 2001 (V7)
  Representation of various optical illusions that are due to the lightness illusion.
  
  
• D.-E. Nilsson and S. Pelger, "A pessimistic estimate of the time required for an eye to evolve", 1994 (V8)
  This paper estimates how long it took evolution to create a proper eye. 
  
  
• J. Koenderink et al., "Picasso in the mind's eye of the beholder: three-dimensional filliing-in of ambiguous line drawings", 2012 (V9)
  Attempt by a well-known Computer Vision scientist to explain how we perceive 3D shapes in Picasso's drawings.
  
  
• F. Cole et al., "Where do people draw lines?", 2008 (V10)
  The authors investigate which lines artists draw to represent 3D objects as pencil drawings.
  
  
• T. Judd et al., "Apparent Ridges for Line Drawing", 2007 (V11)
  Algorithm that uses a 3D model to decide which lines are important for a drawing.   
  
  
• D. Hoffman, "The Construction of Visual Reality", 2011 (V12)
  A neuroscientist's rationale for why we see optical illusions and why they may have been evolutionarily advantageous.  
   
• M. Bach, "Die Hermann-Gitter-Täuschung: Lehrbucherklärung widerlegt", 2008 (V13)
  Interesting experiment on the Hermann grid, which impressively refutes the previous explanation of lateral inhibition alone. (DE) 
  
  
• J. Faubert and A. Herbert, "The peripheral drift illusion: A motion illusion in the visual periphery", 1999 (V14)
  Explanation of motion illusions due to gray value gradients, basis of the Rotating Snakes illusion.
  
  
• F. Poirier, The Anderson-Winawer illusion: it's not occlusion", 2009 (V15)
  Observations and attempt to explain the lightness illusion. Additional paper on the topic:
  •  "Clouds are not normal occluders, and other oddities", 2012 (V15.1)
    
    
• R. Masland and P. Martin, "The unsolved mystery of vision", 2007 (V16)
  Perceptual aspects that we cannot yet explain.
  
  
• D. Simons and R. Rensink, "Change blindness: Past present and future. Trends in Cognitive Sciences", 2005 (V17)
  About what we see but do not perceive.
  
  
• R. Conway et al., "Neural Basis for a powerful static motion illusion", 2005 (V18)
  Attempt to explain the Rotating Snakes illusion, see also:
  • A. Kitaoka, "Anomalous motion illusion and stereopsis", 2006 (V18.1)
    
    
• J. Koenderink et al., "Pointing out of the picture", 2004 (V19)
  Fascinating investigation into the perception of two-dimensional projection of three-dimensional scenes including an explanation of why some eyes in paintings seem to follow us.
  
  
• P. Cavanagh et al., "View dependence of 3D recovery from folded pictures and warped 3D faces", 2004 (V20)
  Fun with pictures of faces.     
  
  
• T. Oskam et al., "OSCAM - Optimized Stereoscopic Camera Control for Interactive 3D", 2011 (V21)                                                                                              Disney Research presents a real-time solution to preserve stereoscopic perception when viewing 3D
  
  
• Lang et al., "Nonlinear Disparity Mapping for Stereoscopic 3D", 2010 (V22)                                                                                                                              Another paper from Disney allowing for retargeting and warping content to preserve depth perception depending on the projection media
  
  

Images and the Mind

• B. Nanay, "Picture perception and the two visual subsystems", 2008 (M1)
  Possible importance of the brain's two image processing pathways for image perception.
  
  
• B. Piqueras-Fiszman and C. Spence, "The Influence of the color of the cup on consumers' perception of a hot beverage", 2012 (M2)
  Answer to the question of which cocoa mug tastes best.  
       
  
• R. Russell,"A sex difference in facial contrast and its exaggeration by cosmetics", 2009 (M3)
  Why and how makeup works.

• J. Coan and J. Gottman, "The Specific Affect Coding System (SPAFF)", 2007 (M4)
  System for describing facial expressions for the purpose of emotion analysis.

• M. Jensen, K. Mathewson, "Simultaneous perception of both interpretations of ambiguous figures", 2011 (M5)
  Discovery of how we can make our brain see both motifs at the same time in ambiguous images.        
  

• P. Vuilleumier et al., "Distinct spatial frequency sensitivities for processing faces and emotional expressions", 2003 (M6)
  How images can subconsciously trigger emotions.        
  

• P. Vuilleumier et al., "How brains beware: neural mechanisms of emotional attention", 2005 (M7)
  Influence of the unconscious/emotionally perceived on our conscious perception.        
  

• P. Bressloff, "What Geometric Visual Hallucinations Tell Us about the Visual Cortex", 2002 (M8)
  Paper proposing an explanation for patterns hallucinated under the influence of drugs based on the functioning of the visual cortex.        
  

• J. Pettigrew and S. Miller, "A sticky interhemispheric switch in bipolar disorder?", 1998 (M9)
  Influence of manic-depressive (bipolar) disorder on visual perception.

• K. Koscinski, "Current status and future direction of research on facial attractiveness", 2009 (M10)
  State of the art research on what makes a face attractive, see also:
  - G. Rhodes, "The evolutionary psychology of facial beauty", 2006 (M10.1)  

• F. Strack et al., "Inhibiting and facilitating conditions of the human smile", 1988 (M14)
  When our facial muscles are forced to laugh, we judge image content more positively than when we look sad.        
  

• D. Maurer et al., "The shape of boubas: sound-shape correspondences in toddlers and adults", 2006 (M15)
  Investigation of the Bouba/Kiki effect in infants and adults.

• M. Ernest-Jones et al., "Effects of eye images on everyday cooperative behavior: a field experiment", 2010 (M16)
  How we change our behavior when we (subconsciously) feel observed.    
  
  

Images and Art

• S. Littler, "A Linear Perspective to Art", 2004  (A1)
  Exciting historical development of central projection in paintings.

• Jacci Howard Bear, "Create a 3D Photo Effect With GIMP", 2020  (A2)
  Instructions on how to use GIMP to add depth to ordinary images (this topic/tutorial should be presented as a a live demo or by explaining why each image transformation contributes to building up the 3D effect).   
  

• A. Kalaidjian, "Automated landscape painting in the style of Bob Ross", 2009 (A3)
  How to paint (kitschy) landscapes without being able to paint.
  

• P. Cavanagh, "The artist as neuroscientist", 2005 (A4)
  Fascinating view of fine art by a neuroscientist.           
  

• V. Ramashandran and W. Hirstein, "The science of art: A neurological theory of aesthetic experience", 1999 (A5)
  Neuroscientific considerations and attempts to explain various aspects of art and why humans like art.

• M. Petry, "Sculpture and Touch", 2008 (A6)
  Examination, from an art perspective, of the differences between the senses of touch and sight using sculptures.

• B. Pinna, "The organization of shape and color in vision and art", 2011(A7)
  Gestalt psychological aspects in art.

• M. Hooser, "The Language of Art: A covnersation between Henri Matisse and Pablo Picasso", 2001 (A8)
  Comparison of the styles of Matisse and Picasso from an artistic point of view.

• H. Cohen, "What is an image?", 1979  (A10)
  An attempt to answer what an image is from the point of view of an AI scientist. Gives the description of an "art robot".
  
  
• R. Pepperell, "Connecting Art and the Brain: An Artist's Perspective on Visual Indeterminacy", 2011, (A11)
  
  
• R. Pepperell, "The perception of art and the science of perception", 2012, (A12)
  
  

Further Reading:

• J. Frisby and J. Stone, "Seeing - the computational approach to biological vision", MIT Press, 2010
  
  
• J. Su, D. V. Vargas and K. Sakurai,"One pixel attack for fooling deep neural networks", 2018
  
  
• Donald Hoffman: Visual Intelligence. Norton, 1998.
  - Simon Ings: A Natural History of Seeing. Norton, 2007.
  - Patrick Cavanagh: The Artist as Neuroscientist. Nature, vol. 434, March 2005.

Agenda: