This lecture
Graphics concepts and considerations for the web
Surfaces and meshes
Sketchfab & interactive 3D movies online for planet and galaxy simulations
3D Computer Graphics
Composed of virtual 3D objects
Often time evolving (animated)
Displayed on 2D screens
Attempts to simulate photorealism to some extent
note these can also be displayed on 3D screens, but the way that happens is a little different
Computer Graphics Terms
Real-time vs Pre-Rendered graphics
Frames vs Timesteps
real-time graphics refresh the screen faster than the eye perceives, usually at least 30 times a second. Pre-rendered can take all the time in the world.
Frames are individual images that when strung together in time create the illusion of motion. They are the "timesteps" of a movie. But scientific data also have "timesteps" which may not be synchronized with the speed of the movie.
You'll notice in this GIF, the frame rate and the time step rate are different - we are zooming out quicker than we are updating the data in the images
Computer Graphics Acronyms
VFX - visual effects
CGI - computer graphics imagery
FPS - frames per second
GUI - graphic user interface
HUD - heads up display
you aren't required to remember any of these, but if you see them pop up, here they are
Common Frame Rates
24 FPS - theatrical films
30 FPS - TV and specialty theaters
48 FPS, 60 FPS - video games, interactive graphics, virtual reality
120 FPS - really good virtual reality
VIDEO
you can note if you try to follow the 15 or 30 FPS (frames per second) with your eye you see that it is jumpy
24 fps is considered the absolute minimum necessary frequency so that people don't perceive individual frames.
48 fps is widely considered so fast the human eye can't perceive any separation at all - but 60 fps is safer.
if you watch a TV and it seems distractingly smooth, it's probably doing frame interpolation to make 30fps content play back at 60fps.
This youtube video actually only plays at 60FPS, so the 120 ball is kind of pointless.
3D Geometry
All 3D geometry is represented as:
3D Geometry
All 3D geometry is represented as:
3D Geometry
All 3D geometry is represented as:
3D Geometry
All 3D geometry is represented as:
Points
Edges
Surfaces
Volumes
or we can give the 3D surfaces some depth and make them into shapes
here these 3D cubes are called "voxels" which are similar to 2D "pixels" but for volumes
3D Geometry
Surfaces can be encoded as:
primitives are defined by mathematical functions. This sphere is defined by a center position and a radius.
3D Geometry
Surfaces can be encoded as:
Implicit primitives
Polygonal Meshes
Quadrilaterals are a good way to see the flow of geometry, which artists like, but quads can bend.
3D Geometry
Surfaces can be encoded as:
Implicit primitives
Polygonal Meshes
Triangles cannot bend because three points define a plane. So automatic geometry like the stuff you use in science is more often going to be triangles.
3D Geometry
Surfaces can be encoded as:
Implicit primitives
Polygonal Meshes
NURBS or Bezier Surfaces
This is a NURBS sphere. You can see the control vertices are floating off the surface. Every patch on the surface is influenced by many of the neighboring points.
Here if I wanted to deform this surface, I could pull at one of the blue points and the shape of the surface would change.
3D Geometry
Surfaces can be encoded as:
Implicit primitives
Polygonal Meshes
NURBS or Bezier Surfaces
Subdivision Surfaces
Subdivision surfaces are like those adaptive volumes we saw last week. You can add detail where you want it.
3D Geometry
Datasets with many fields called "attributes":
Transform Attributes (translate, rotate, scale)
These transform attributes are the same for all objects.
Recall we discussed some of these in earlier classes for 2D plots. Same principle here.
3D Geometry
Datasets with many fields called "attributes":
Transform Attributes (translate, rotate, scale)
Shape Attributes (radius, bumpyness, twistyness)
shape attributes depend totally on what the shape is.
These are something you won't really see in 2D unless you start modifying 2D shapes with some weird transforms.
OpenGL
Before most computers could show graphics of any kind, several companies began to compete for proprietary formats.
A company called Silicon Graphics stepped in and created an Open Source specification for computer graphics called OpenGL.
To this day, most software graphics you see are rendered using some version of OpenGL - including the whole Mac operating system.
Many other open source projects have copied this model, such as OpenCV (computer vision), OpenCL (gpus), and OpenMP (multi-processing).
WebGL
OpenGL is primarily intended for C-style programming.
WebGL implements the same set of tools for rendering through a web browser. This allows us to natively render 3D content on the internet!
SketchFab & ipyvolume
SketchFab uses WebGL to render YOUR 3D data in a 3D viewport in a web browser.
SketchFab.com
ipyvolume uses WebGL to render volumes to your jupyter notebook.
Rendering in 3D: From 3D objects to images
Our task is to take our physical description of 3D space and convey to the computer how to bounce light around.
Rendering in 3D: From 3D objects to images
Our task is to take our physical description of 3D space and convey to the computer how to bounce light around.
For example:
Image Credit: ALMA & Erik Rosolowsky
in this image there are layers of transmission and absorption (a transmission function encodes this) which added together make this volume rendering
Rendering in 3D: From 3D objects to images
Our task is to take our physical description of 3D space and convey to the computer how to bounce light around.
We talked about this in abstract but today we'll get practicle:
learn how to turn our numerical simulations into 3D geometry files
make aesthetic choices motivated by physics
load our models into MeshLab to "debug"
load final(ish) models into Sketchfab to make 3D movies
Software
Setup a Sketchfab account
Download MeshLab
Planets - Texture Mapping
we will do something called texture mapping where we "wrap" a high resolution image on a low resolution mesh object
here you can see how this works for a face, and it indeed looks creepy
Planets - Texture Mapping
we will do this with our planets as we will have very low resolution spheres
why? this is actually a graphics trick that is widely used - it turns out is more computationally intensive to render lots of polygons than few polygons and high resolution textures
Planets - Texture Mapping
here are a few of the texture maps that are included with the files you'll download for your 3D movies
we'll go through the code in a moment, but basically, you'll run some code with what sorts of "planets" your planet simulations are and you'll end up with a directory called something like "MyPlanetSystem"
in this directory you will have the planet textures and two files an obj , or object file, and an mtl , or material, file
these files do different things to combine to make your model
your obj file gives vertex information geometric locations where we should "pin" our texture maps
the vertexes tell us where each "square" that makes up our sphere goes - you can see this sort of in this image here
our material files can give us information about background colors (which we won't be using) and also the names of any texture maps we'll be using like "sun_texture1.jpg" for example
you will use these files to import into MeshLab and upload to sketchfab
I highly recommend you look at them in MeshLab first since processing models to Sketchfab takes a while!
when debugging in MeshLab when you first upload you might see some black spheres - to fix this go to the "Render" part of the tool bar and select "Color" and then "Per Mesh"
if you have been doing galaxy simulations you'll also have an option to upload those as well in a different file format called "PLY" which gives the dot locations and their colors
notice that these can only be uploaded on sketchfab so its recommended you try out the planet files first to get the hang of things
if you have tried a few different models and are ready to start making things more complex, you can start adding in physically modivated sizes and planet textures to your viz
for example, depending on the mass of your star, it should have a different size and color as shown by this chart right here
the same thing is true of planets - depending on their mass, they usually have different size
additionally, their mass and size determines what type of planet - is it earth like and therefore you should use a mars/earth texture? Or is it more jupyter like and you should use a jupyter or saturn texture?
is it far out from the star and gaseous? If so, maybe a neptune texture
Let's do some 3D planets with Python!
Warm-Up Activity
What is the visualization trying to show?
What are its methods?
What are the strengths / weaknesses?
https://projects.fivethirtyeight.com/mortality-rates-united-states/
Don't worry - there aren't necessarily right or wrong answers here!
