Before we begin, I want to differentiate between OpenGL and OpenGL ES. Why does Android use OpenGL ES over OpenGL? What does ES stand for?
Well, OpenGL means Open Graphics Library and ES means Embedded System. So OpenGL ES means OpenGL for embedded systems. Android is a very compact and minimal operating system which isn’t capable of computing or processing heavy computation like computers (Gaming computers 🤓).
Moreover, OpenGL ES has fewer features than OpenGL, so some features that work on OpenGL might not work on the ES version. But don’t worry, it has everything to build a complete three-dimensional game, which is pretty cool.
Compatibility in Android
OpenGL ES 1.0 has been added since Android 1.0, and OpenGL ES 2.0 also has been included in Android 2.2. You can find the docs here.
In this tutorial, we will choose OpenGL ES 2.0 since it seems to be stable.
What are we going to do?
In this blog post, In spite of drawing a single triangle, I will guide you through the whole process to make a Photo Editor application which has some basic features such as Filters, Texts overlay and show you how to save it as a bitmap which you can save it into a file or do whatever we want.
If you coming from the Google OpenGL documentation and feel overwhelmed, well you are not alone. The docs are quite too old and difficult to understand for beginners, however it doesn’t mean it isn’t useful at all.
Who should continue reading this?
Before continue reading this post, I encourage you to learn some basic understanding of how OpenGL works, but if you’re lazy, that’s Okay
And of course some basic understanding of Kotlin/Java and Android are needed yet if you’re not good at it that’s okay.
What is OpenGL on Android?
To be clear OpenGL is not a library or something that we can see its source code. It’s just a specification whose code has implemented in the GPU by manufacturers like Nvidia or AMD. The word Open doesn’t mean it’s open-source.
In order to create an OpenGL context, we need EGL and a thread. So what are Context and EGL?
Well, OpenGL Context is where we can call OpenGL functions. OpenGL Context is bound into a thread, so if you have a thread that has the context, but you call OpenGL functions in the different thread like the main thread, it won’t work at all.
In Android, we use EGL to create and configure OpenGL context; however, in different platforms like C/C++ we something else. EGL and OpenGL are being maintained by the Khronos Group.
But why OpenGL?
Well, it’s one of the most widely-used graphic API, so it has rich documentation and tutorials. And it’s used by almost every device.
Why do we need to do image processing on the GPU anyway? The answer is that GPU is very good at computing graphic related stuff. It’s highly optimized and built to do this kind of things such as color processing something that CPU finds it challenging to achieve.
Keywords
- Shader — an OpenGL program. That’s it!
- Vertex Shader — a code that uses to draw each vertex. If we have 5 vertices, it will run 5 times.
- Fragment Shader — aka. (Pixel Shader) — is code that runs each pixel inside your shape. This is where we do a lot of stuff like processing colors of an image.
- GLSL (Shading Language) — A programming language used by OpenGL.
- Texture — a 2D texture can be sent to a shader to display to the screen. It can be an image.
- SurfaceTexture — It’s a texture! But it’s an object while the texture is just an Integer.
- Surface — an object that OpenGL draws into. It’s like a paper, and OpenGL is a pen.
- ByteBuffer — used to transfer byte array from JVM Memory to CPU memory.
- FloatBuffer — used to transfer float array from JVM Memory to CPU memory.
- EGLSurface — A surface that OpenGL draws into but it has been done off-screen and then sends it to the Surface.
- EGLContext — A context which contains OpenGL configurations
- SwapBuffers — The process of transferring data from GPU to display into the screen or a Surface.
- Framebuffer — A place where OpenGL renders to and being swapped to the screen or a surface. This can be used to draw something off-screen.
Creating an OpenGL window
We have two options in order to create an OpenGL window to display what has been drawn on the screen. The first easy option is to use GLSurfaceView which extends from SurfaceView. However, SurfaceView is not a simple view like TextureView, so we cannot overlay other views on top of SurfaceView which makes it not good. As a result, we are going to use TextureView, but we don’t have GLTextureView built-in in the Android framework, so we gotta make it by ourselves. By doing this, you will understand more about how to get started with OpenGL from scratch.
Before creating GLTextureView, we need something called EGLContext which connects our TextureView to the GPU, so let’s begin.
- Create EGLContext
- Create EGLSurface
- Attach our SurfaceTexture to EGLContext
- MakeCurrent() the EGLSurface
- Create an EGL Thread
- Ready to draw something onto the screen
Shaders in OpenGL ES
There plenty of shaders in OpenGL ES, but in this case, we can use only two shaders which are Vertex Shader and Fragment Shader(Pixel Shader).
Let’s draw some shapes to be clear about what it does.
Coordinate System
Its coordinate system is from -1.0 to 1.0, so something that is greater than this value will not be drawn.
Keep in mind that OpenGL can only draw triangles, so when we want to draw something else we have to combine a bunch of triangles together to get a shape we want. So let me explain you!
So in order to draw a rectangle we have to combine two identical triangles together something like the picture below.
And we got a Rectangle. Hopefully, my explanation would make sense to you. Note that the colors on each triangle is just a visual representation only. In the real application, we can set its color the way we want. (That’s Fragment Shader job)
