VR Jester API

Virtual Reality Gesture Recognition API for Minecraft VR

Play Vivecraft or QuestCraft? Love mods? Ever wish mods were more VR compatible? Wait no further. This is a solution for devs and players alike!

This API enables Minecraft mod developers to easily implement support for their mod to be more VR compatible. Create any custom gestures that map to any key binds and trigger events. No longer do you need to map controller keybindings to keyboard keys. With this API, your body is the controller.

__Demos:__

- https://imgur.com/a/UYaVFCj - https://imgur.com/a/sYXcike - https://i.imgur.com/kBD1TIU.mp4 - https://imgur.com/a/0fpiQ9a - https://i.imgur.com/6xUNF6k.mp4

Installation:

1. Install required dependency mods. Note that Questbind is only required for QuestCraft usage. 2. Download the VR Jester API mod & place it in the `mods` folder. 3. Run Minecraft. Necessary config files will generate on initialization.

Usage:

1. While in Minecraft VR, bind one of your controller buttons to the "Gesture Listener Trigger". I recommend a grip button. For Vivecraft you must do this through Steam VR's controller bindings menu. For QuestCraft you do this through QuestBind. 2. These are 2 ways to create gestures: - Using the `/gesture` command - Modifying the `config/gesture_store.json` directly where gestures are stored. Sample gesture store: ```json "GESTURE 1": { "RIGHT_CONTROLLER|LEFT_CONTROLLER": [ { "vrDevice": "RIGHT_CONTROLLER|LEFT_CONTROLLER", "movement": "forward", "elapsedTime": 0, "speed": 0.0, "direction": { "x": 0.0, "y": 0.0, "z": 0.0 }, "devicesInProximity": {} } ] }, ... ``` 3. Once your gestures are created, you can map them to key binds in `config/VRJesterAPI.cfg` by creating key value objects under the field *"GESTURE_KEY_MAPPINGS"*. You can find the key bind names in `.minecraft/options.txt`. *(Note: The actual KeyMapping string of an individual key bind in that file is in between the key_ and colon. i.e.: key_`key.inventory`:18)* The *"KEY_ACTION"* field determines whether the recognized gesture triggers a single *click* key press or a *hold* down key press. A *hold* is triggered when you keep your gesture in place after it's recognized. ```json "GESTURE_KEY_MAPPINGS": { "GESTURE 1": { "KEY_MAPPING": "examplemod.key.ability_1", "KEY_ACTION": "click" }, "GESTURE 2": { "KEY_MAPPING": "key.hotbar.2,key.sneak", "KEY_ACTION": "hold" }, "GESTURE 3": { "KEY_MAPPING": "key.keyboard.j", "KEY_ACTION": "click" } } ``` [modrinth]: https://img.shields.io/badge/Modrinth-1bd96a?logo=modrinth&logoColor=ffffff [curseforge]: https://img.shields.io/badge/CurseForge-f16436?logo=curseforge&logoColor=ffffff

Download: [ ![modrinth] ](https://modrinth.com/mod/vrjesterapi) [ ![curseforge] ](https://curseforge.com/minecraft/mc-mods/vrjesterapi)

Info:

- Consider each object within the square brackets [ ] a mere *piece* of the gesture. You can add multiple of these GestureComponent objects. This API recognizes gestures as complex as you want! Just know the more conplex, the more difficult it is to perform a gesture correctly to match your stored gesture. - Different values for *"movement"* are `forward, back, left, right, up, down`. The movement direction is relative to your facing direction at the moment you click the Jester Trigger which initiates the gesture listener. Simple example is punching in the same direction you're facing would be recognized as "forward". - Different values for *"vrDevice"* are `RIGHT_CONTROLLER, LEFT_CONTROLLER, HEAD_MOUNTED_DISPLAY` and if you want a gesture recognized on multiple VR devices you can incorporate a 'logical or' using a pipe `|` like so `RIGHT_CONTROLLER|LEFT_CONTROLLER|HEAD_MOUNTED_DISPLAY` - The *"elapsedTime"* is in milliseconds. So putting a value of `2000` would mean that part of the gesture lasts 2 seconds. This would be good for a charge up sort of move. - The *"speed"* field is a float value representing the velocity of that GestureComponent. A decent threshold to recognize the speed of a punch would be `1500.0-2000.0` and feel free to play around with the values as much as you want. - The *"direction"* field takes `x,y,z` float values that create a 3D normalized vector. Example usage of this would be having `{0.0, 1.0, 0.0}` to recognize when the player's brick hand is facing upwards. There's a certain threshold so it doesn't have to be *exact*. - Finally there's *"devicesInProximity"* which is formatted like so: ```json "devicesInProximity": { "LEFT_CONTROLLER": 20 } ``` What this example means is that the vrDevice of this GestureComponent has to be within proximity of the left controller for 20 ticks. - Subscribe to the event bus to handle events triggered by a recognized gesture (Forge) or register to the event callback interface (Fabric).

Forge: ```java @SubscribeEvent public void onGestureEvent(GestureEvent event) { // gesture handler code here -> event.getGestureName() } ```

Fabric: ```java public static void init() { GestureEventCallback.EVENT.register((gestureEvent) -> { // gesture handler code here -> gestureEvent.getGestureName() }); } ```

Planned:

In the future I'll add continous gesture recognition, more configuration options, and optimizations.

*Update 3/27/2025* - I know I haven't updated in a while, but I fully intend to come back to this mod. I've been working on other projects. Apologies for the inconvenience.