Overview

The OCR Auto Clicker JS API provides a set of JavaScript interfaces for automating operations on Android devices. Through the window.mumudroid object, you can perform screen clicks, swipes, app launching, and text-recognition-based automation tasks.

⚠️ Security Warning: Running untrusted scripts can compromise your personal privacy or lead to account bans. Please ensure you only run code from reliable sources, or code that you have written or fully understand yourself.

Coordinate System Description

All APIs involving coordinates support two positioning methods:

  • Relative Proportional Positioning (Recommended): Coordinate values are between 0 and 1, representing the relative position on the screen (e.g., 0.5 represents the exact center). This method perfectly adapts to devices with different resolutions.
  • Absolute Coordinate Positioning: Takes effect when coordinate values are greater than 1, representing absolute pixel positions on the screen (e.g., 500 represents the 500th pixel).

1. Standard Script Structure Template

To ensure the script executes asynchronously (such as using sleep for delays and continuous looping), it is recommended to use an IIFE (Immediately Invoked Function Expression) combined with async/await as the basic skeleton of your script:

(function() {
  'use strict';
  
  // Define the asynchronous main loop function
  async function loop() {
    // 1. Register global callbacks (e.g., text search result callback)
    // window.onFindTextResult = ...

    // 2. Start the infinite monitoring loop
    while (true) {
      // Must include sleep to prevent the main thread from deadlocking
      await sleep(1000); 
      
      // Write your automation logic here
      console.log("Script is running...");
    }
  }
  
  // Start the script
  loop();
})();

2. Core Event Callbacks

window.onFindTextResult

Function Description: This is a global callback function used to receive the asynchronous search results of the find_text() method. Once the underlying engine finishes recognizing text on the screen, this function will be triggered automatically.

Parameter Description:

Parameter Type Description
result boolean Whether the target text was successfully found (true if found, false if not found)
text string The actual matched text content
centerX number The X coordinate of the matched text's center point on the screen (can be passed directly to the click function)
centerY number The Y coordinate of the matched text's center point on the screen (can be passed directly to the click function)

Usage Example:

window.onFindTextResult = function(result, text, centerX, centerY) {
    if (result) {
        console.log(`Found text: ${text}, preparing to click center point: ${centerX}, ${centerY}`);
        // Automatically click the found text
        click(centerX, centerY);
    } else {
        console.log("Target text not found on current screen");
    }
};

3. Detailed API Description

3.1 sleep(ms)

Function Description: Asynchronously pauses script execution for a specified number of milliseconds, used for waiting for page loads or animations to finish.

  • Parameter: ms (number) - The number of milliseconds to pause.
  • Example: await sleep(2000); // Pause for 2 seconds

3.2 click(x, y, [duration])

Function Description: Clicks on a specified screen coordinate. Supports both normal clicks and long presses.

  • Parameters:
    • x, y (number): Target coordinates.
    • duration (number, optional): Press and hold duration (in milliseconds). If not provided, the system handles the default click duration.
  • Example: click(0.5, 0.5); // Click the center of the screen

3.3 clickBack() / clickHome()

Function Description: Simulates pressing the Android system's "Back" button or "Home" button.

  • Example: clickBack();

3.4 swipe(x1, y1, x2, y2, [duration])

Function Description: Performs a screen swipe operation.

  • Parameters:
    • x1, y1 (number, default 0.5, 0.8): Starting point coordinates.
    • x2, y2 (number, default 0.5, 0.2): Ending point coordinates.
    • duration (number, optional): Swipe duration (in milliseconds). If not provided, a random duration between 300~500ms is generated.
  • Example: swipe(0.5, 0.8, 0.5, 0.2); // Swipe up the screen

3.5 find_text(text, [x1, y1, x2, y2])

Function Description: Asynchronously searches for text within the screen (or a specified area).

💡 Advanced Feature: Supports using an English comma , to separate multiple keywords. The search is considered successful as long as any one of the keywords appears on the screen.

  • Parameters:
    • text (string): The text to search for, supports multiple keywords (e.g., 'skip,跳过,跳過').
    • x1, y1, x2, y2 (number, optional): Defines the rectangular search area (relative proportion 0~1), defaulting to full screen (0,0,1,1).
  • Example: find_text('Confirm,OK,Sure');

3.6 openApp(packageName)

Function Description: Launches an application via its package name.

  • Example: openApp('com.tencent.mm'); // Launch WeChat

3.7 openScheme(schemeUrl)

Function Description: Launches an app or navigates to a specific page via DeepLink / Scheme protocol.

  • Example: openScheme('taobao://...');

4. Practical Example: Auto Skip Ads / Splash Screens

Below is a complete practical script, commonly used to automatically recognize and click the "Skip" button on various apps. It combines the standard template, multi-keyword search, and asynchronous callback mechanisms.

(function() {
  'use strict';
  
  // ⚠️ Warning: Running untrusted scripts can compromise your personal privacy. 
  // Only run code from sources you trust.

  async function loop() {
    // 1. Register text search result callback
    window.onFindTextResult = function(result, text, centerX, centerY) {
        console.log("onFindTextResult: " + result + " " + text + " " + centerX + " " + centerY);
        
        // 2. If the target text is found, directly click its center coordinate
        if (result) {
          click(centerX, centerY);
          console.log("Successfully clicked the skip button!");
        }
    };

    // 3. Start the infinite monitoring loop
    while (true) {
       // Scan the screen every 1 second (to prevent excessive performance consumption)
       await sleep(1000);
       
       console.log("Scanning screen...");
       
       // 4. Initiate text search (supports comma-separated multiple keywords in English/Chinese)
       // The system will recognize text in the background and trigger onFindTextResult upon completion
       find_text('skip,跳过,跳過,close ad');
    }
  }
  
  // Start the main loop
  loop();
})();

Example Logic Analysis:

  • Anti-Deadlock Mechanism: The while(true) loop must include await sleep(1000), otherwise it will cause the JS engine to deadlock and the app to freeze.
  • Asynchronous Decoupling: find_text only initiates a search request; the actual processing logic is written in window.onFindTextResult. This design avoids blocking the main thread.
  • Precise Clicking: The centerX and centerY returned by the callback are the geometric center points of the text calculated by the engine. Passing them directly to click() ensures the highest click success rate.
  • Multi-Language / Multi-Scenario Compatibility: 'skip,跳过,跳過' utilizes the interface's comma-separated feature, covering English, Simplified Chinese, and Traditional Chinese skip button texts all at once.

5. Notes and Best Practices

  • Asynchronous Callback Awareness: Remember that find_text() does not return results synchronously. Do not write if statements immediately after find_text(); you must rely on the window.onFindTextResult callback.
  • Reasonable Use of Sleep: After click, swipe, or openApp, it is recommended to add await sleep(1000~3000) to give the system enough time to render the next page. Otherwise, the subsequent find_text might capture content from the old page.
  • Regional Search for Performance Optimization: If you clearly know that the "Skip" button only appears in the top right corner of the screen, you can use find_text('skip', 0.7, 0, 1, 0.2) to limit the search area. This will significantly boost OCR recognition speed and reduce CPU usage.
  • Silent Environment Protection: All APIs internally contain the if (!window.mumudroid) return; check. If you test the script in a standard web browser environment, the interfaces will not throw errors, but they will not execute actual operations either.