Display and control your Android device
Update DEVELOP.md
Romain Vimont5ffdcbb7beea
diff --git a/DEVELOP.md b/DEVELOP.mdindex ba9e130b..dea8137d 100644--- a/DEVELOP.md+++ b/DEVELOP.md@@ -32,7 +32,7 @@ The server is a Java application (with a [`public static void main(String...args)`][main] method), compiled against the Android framework, and executed as`shell` on the Android device.-[main]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/Server.java#L100+[main]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Server.java#L123To run such a Java application, the classes must be [_dexed_][dex] (typically,to `classes.dex`). If `my.package.MainClass` is the main class, compiled to@@ -65,18 +65,18 @@ They can be called using reflection though. The communication with hiddencomponents is provided by [_wrappers_ classes][wrappers] and [aidl].[hidden]: https://stackoverflow.com/a/31908373/1987178-[wrappers]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers-[aidl]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/aidl/android/view+[wrappers]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers+[aidl]: https://github.com/Genymobile/scrcpy/tree/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/aidl/android/view### Threading-The server uses 2 threads:+The server uses 3 threads:- the **main** thread, encoding and streaming the video to the client;- - the **controller** thread, listening for _control events_ (typically,- keyboard and mouse events) from the client.- - the **receiver** thread (managed by the controller), sending _device events_+ - the **controller** thread, listening for _control messages_ (typically,+ keyboard and mouse events) from the client;+ - the **receiver** thread (managed by the controller), sending _device messges_to the clients (currently, it is only used to send the device clipboardcontent).@@ -92,9 +92,9 @@ The video is encoded using the [`MediaCodec`] API. The codec takes its inputfrom a [surface] associated to the display, and writes the resulting H.264stream to the provided output stream (the socket connected to the client).-[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java+[`ScreenEncoder`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java[`MediaCodec`]: https://developer.android.com/reference/android/media/MediaCodec.html-[surface]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L69-L70+[surface]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L68-L69On device [rotation], the codec, surface and display are reinitialized, and anew video stream is produced.@@ -108,31 +108,30 @@ because it avoids to send unnecessary frames, but there are drawbacks:Both problems are [solved][repeat] by the flag[`KEY_REPEAT_PREVIOUS_FRAME_AFTER`][repeat-flag].-[rotation]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90-[repeat]:-https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148+[rotation]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L90+[repeat]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java#L147-L148[repeat-flag]: https://developer.android.com/reference/android/media/MediaFormat.html#KEY_REPEAT_PREVIOUS_FRAME_AFTER### Input events injection-_Control events_ are received from the client by the [`EventController`] (run in-a separate thread). There are several types of input events:+_Control messages_ are received from the client by the [`Controller`] (run in a+separate thread). There are several types of input events:- keycode (cf [`KeyEvent`]),- text (special characters may not be handled by keycodes directly),- mouse motion/click,- mouse scroll,- other commands (e.g. to switch the screen on or to copy the clipboard).-All of them may need to inject input events to the system. To do so, they use-the _hidden_ method [`InputManager.injectInputEvent`] (exposed by our+Some of them need to inject input events to the system. To do so, they use the+_hidden_ method [`InputManager.injectInputEvent`] (exposed by our[`InputManager` wrapper][inject-wrapper]).-[`EventController`]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/EventController.java#L66+[`Controller`]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/Controller.java#L81[`KeyEvent`]: https://developer.android.com/reference/android/view/KeyEvent.html[`MotionEvent`]: https://developer.android.com/reference/android/view/MotionEvent.html[`InputManager.injectInputEvent`]: https://android.googlesource.com/platform/frameworks/base/+/oreo-release/core/java/android/hardware/input/InputManager.java#857-[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/v1.8/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27+[inject-wrapper]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/server/src/main/java/com/genymobile/scrcpy/wrappers/InputManager.java#L27@@ -188,12 +187,13 @@ The client uses 4 threads:- the **main** thread, executing the SDL event loop,- the **stream** thread, receiving the video and used for decoding andrecording,- - the **controller** thread, sending _control events_ to the server.+ - the **controller** thread, sending _control messages_ to the server,- the **receiver** thread (managed by the controller), receiving _device- events_ from the client.+ messages_ from the client.In addition, another thread can be started if necessary to handle APK-installation or file push requests (via drag&drop on the main window).+installation or file push requests (via drag&drop on the main window) or to+print the framerate regularly in the console.@@ -217,10 +217,10 @@ to decode a new frame while the main thread renders the last one.If a [recorder] is present (i.e. `--record` is enabled), then its muxes the rawH.264 packet to the output video file.-[stream]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/stream.h-[decoder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/decoder.h-[video_buffer]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/video_buffer.h-[recorder]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/recorder.h+[stream]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/stream.h+[decoder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/decoder.h+[video_buffer]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/video_buffer.h+[recorder]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/recorder.h```+----------+ +----------+@@ -234,19 +234,19 @@ H.264 packet to the output video file.### Controller-The [controller] is responsible to send _control events_ to the device. It runs-in a separate thread, to avoid I/O on the main thread.+The [controller] is responsible to send _control messages_ to the device. It+runs in a separate thread, to avoid I/O on the main thread.On SDL event, received on the main thread, the [input manager][inputmanager]-creates appropriate [_control events_][controlevent]. It is responsible to+creates appropriate [_control messages_][controlmsg]. It is responsible toconvert SDL events to Android events (using [convert]). It pushes the _control-events_ to a queue hold by the controller. On its own thread, the controller-takes events from the queue, that it serializes and sends to the client.+messages_ to a queue hold by the controller. On its own thread, the controller+takes messages from the queue, that it serializes and sends to the client.-[controller]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/controller.h-[controlevent]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/control_event.h-[inputmanager]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/input_manager.h-[convert]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/convert.h+[controller]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/controller.h+[controlmsg]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/control_msg.h+[inputmanager]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/input_manager.h+[convert]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/convert.h### UI and event loop@@ -257,10 +257,9 @@ thread.Events are handled in the [event loop], which either updates the [screen] ordelegates to the [input manager][inputmanager].-[scrcpy]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c-[event loop]:-https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/scrcpy.c#L187-[screen]: https://github.com/Genymobile/scrcpy/blob/v1.8/app/src/screen.h+[scrcpy]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c+[event loop]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/scrcpy.c#L201+[screen]: https://github.com/Genymobile/scrcpy/blob/ffe0417228fb78ab45b7ee4e202fc06fc8875bf3/app/src/screen.h## Hack