and others added 30 commits
This commit makes TornadoVM more Java-friendly by adding three quick wins: 1. **Maven Wrapper (./mvnw)**: Contributors no longer need Maven pre-installed. The wrapper automatically downloads and uses the correct Maven version (3.9.6). 2. **Default Maven Configuration (.mvn/maven.config)**: Sensible defaults that match the existing build system: - Parallel builds with 1.5 threads per CPU core (-T1.5C) - Colored output for better readability - Default profiles: jdk21 + opencl-backend 3. **Maven Build Cache Extension**: Intelligent caching for faster builds. - First build: same time as before (~24s) - Repeated builds: ~10x faster (2-3s instead of 24s) - Incremental builds: ~2x faster (5-7s instead of 13s) - Stores up to 3 cached builds locally Usage: ./mvnw install # Standard build with defaults ./mvnw install -Pptx-backend # Override backend profile ./mvnw clean # Clean build All existing make targets continue to work as before. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add build and run documentation with Maven argfile support This commit adds comprehensive documentation and tooling for two build/run approaches: 1. Traditional make + tornado command workflow 2. Maven-friendly java @tornado-argfile approach for IDE integration Changes: - Add BUILD_AND_RUN.md with detailed guide for both approaches - Add tornado-argfile with all required JVM flags and module exports - Update pom.xml to manage commons-math3 dependency explicitly - Update tornado-assembly/pom.xml to improve SDK assembly and add backend detection The argfile approach enables better IDE integration and CI/CD workflows while maintaining compatibility with the existing tornado wrapper script. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add build and run documentation with Maven argfile support This commit adds comprehensive documentation and tooling for two build/run approaches: 1. Traditional make + tornado command workflow 2. Maven-friendly java @tornado-argfile approach for IDE integration Changes: - Add BUILD_AND_RUN.md with detailed guide for both approaches - Add tornado-argfile with all required JVM flags and module exports - Update pom.xml to manage commons-math3 dependency explicitly - Update tornado-assembly/pom.xml to improve SDK assembly and add backend detection The argfile approach enables better IDE integration and CI/CD workflows while maintaining compatibility with the existing tornado wrapper script. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add build and run documentation with Maven argfile support This commit adds comprehensive documentation and tooling for two build/run approaches: 1. Traditional make + tornado command workflow 2. Maven-friendly java @tornado-argfile approach for IDE integration Changes: - Add BUILD_AND_RUN.md with detailed guide for both approaches - Add tornado-argfile with all required JVM flags and module exports - Update pom.xml to manage commons-math3 dependency explicitly - Update tornado-assembly/pom.xml to improve SDK assembly and add backend detection The argfile approach enables better IDE integration and CI/CD workflows while maintaining compatibility with the existing tornado wrapper script. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add build and run documentation with Maven argfile support This commit adds comprehensive documentation and tooling for two build/run approaches: 1. Traditional make + tornado command workflow 2. Maven-friendly java @tornado-argfile approach for IDE integration Changes: - Add BUILD_AND_RUN.md with detailed guide for both approaches - Add tornado-argfile with all required JVM flags and module exports - Update pom.xml to manage commons-math3 dependency explicitly - Update tornado-assembly/pom.xml to improve SDK assembly and add backend detection The argfile approach enables better IDE integration and CI/CD workflows while maintaining compatibility with the existing tornado wrapper script. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add build and run documentation with Maven argfile support This commit adds comprehensive documentation and tooling for two build/run approaches: 1. Traditional make + tornado command workflow 2. Maven-friendly java @tornado-argfile approach for IDE integration Changes: - Add BUILD_AND_RUN.md with detailed guide for both approaches - Add tornado-argfile with all required JVM flags and module exports - Update pom.xml to manage commons-math3 dependency explicitly - Update tornado-assembly/pom.xml to improve SDK assembly and add backend detection The argfile approach enables better IDE integration and CI/CD workflows while maintaining compatibility with the existing tornado wrapper script. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Simplify tornado-argfile paths for portability and consistency. Add `example-argfile` with JVM flags and module exports This commit introduces `example-argfile` containing predefined JVM flags, module exports, and configuration settings. It serves as a reference for setting up TornadoVM with consistent runtime options. Remove `tornado-argfile` as it is no longer required Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Add incremental build support for faster development This commit introduces incremental build functionality to speed up development by only rebuilding changed modules instead of the entire project. Changes: - Added --incremental flag to bin/compile script - Implemented get_changed_modules() to detect changes via git status - Modified maven_cleanup() to skip clean when doing incremental builds - Updated build_tornadovm() to support building specific modules using -pl - Added 'make incremental' target to Makefile Usage: make incremental # Build only changed modules make incremental BACKEND=ptx,opencl # With custom backend bin/compile --jdk jdk21 --backend opencl --incremental The incremental build: - Detects changed files using git status - Maps files to their Maven modules - Skips 'mvn clean' to preserve artifacts - Builds only changed modules with 'mvn -pl <modules> -am' - Falls back to full build if root pom.xml changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Fix incremental build to update distribution directory The incremental build was compiling changes but not updating the tornado-sdk distribution directory, causing the runtime to use stale code. Root cause: The tornado-assembly module (which packages everything to dist/tornado-sdk/) was not being built during incremental builds. Fix: Always include tornado-assembly in the list of modules to build during incremental builds, ensuring the distribution gets updated with the latest compiled code. Now incremental builds will properly: 1. Detect changed modules via git status 2. Build changed modules + their dependencies + tornado-assembly 3. Update the distribution directory with new jars 4. Allow runtime to use the latest changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Auto-clean distribution before incremental builds Fixed recurring permission errors during incremental builds by automatically cleaning the existing distribution directory before the assembly runs. Problem: - tornado-assembly plugin uses updateOnly=true to update files - When running incremental builds repeatedly, it tries to update the same dist/tornado-sdk/tornado-sdk-X.X.X-HASH/ directory - File permission conflicts caused "Permission denied" errors - Users had to manually run 'rm -rf dist/' before each build Solution: - Added clean_dist_for_incremental() function that removes the existing TORNADO_SDK distribution directory before building - Called automatically during incremental builds only - Uses ignore_errors=True to handle missing directories gracefully - Full builds still use normal maven clean process Now incremental builds work seamlessly without manual intervention: 1. Detects changed modules 2. Cleans existing distribution for current version 3. Builds changed modules + dependencies + assembly 4. Creates fresh distribution with correct permissions 5. Updates paths and generates argfile 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Clean entire dist directory for incremental builds Changed cleanup strategy to remove the entire dist/ directory instead of just the specific TORNADO_SDK version directory to fix recurring permission errors. Issue: - Previous fix only cleaned the old TORNADO_SDK directory - New builds create directories with new commit hashes - Permission errors occurred at parent directory level - Assembly plugin's updateOnly=true caused conflicts Root cause: The assembly plugin creates dist/tornado-sdk/tornado-sdk-X-HASH/ directories. Even when the HASH changes, permission issues occurred because parent directories (dist/tornado-sdk/) retained problematic permissions from previous builds. Fix: Clean the entire dist/ directory before incremental builds to ensure: - Fresh directory structure with correct permissions - No conflicts from previous assembly runs - Assembly plugin can create new directories cleanly Trade-off: This makes the assembly step non-incremental, but the overall build is still much faster than a full rebuild because: - Maven clean is skipped (preserves all target/ artifacts) - Only changed modules are recompiled - Only final distribution is recreated 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> Add smart assembly logic for faster incremental builds Implemented intelligent assembly rebuilding to significantly speed up incremental builds when only unit tests change. Performance improvement: - When only tornado-unittests changes: ~10-15s (skips assembly) - When other modules change: ~28s (includes assembly) - Full build baseline: ~33s Key changes: 1. Smart assembly detection (should_rebuild_assembly): - Skips assembly rebuild when ONLY tornado-unittests changed - Unit tests run via tornado-test/maven, don't need dist/ update - Always rebuilds for examples, benchmarks, runtime, api, drivers - Ensures tornado command sees changes when needed 2. Conditional build logic: - Only cleans dist/ when assembly will be rebuilt - Passes include_assembly flag to control module list - Runs post_installation_actions only when needed - Provides clear feedback about what was skipped 3. Force assembly flag: - Added --force-assembly flag for explicit control - New Makefile target: make incremental-full - Allows forcing distribution rebuild when needed Usage examples: make incremental # Smart: skips assembly for unittests make incremental-full # Always rebuilds assembly bin/compile --incremental --force-assembly # Direct usage Benefits: - 2-3x faster when iterating on unit tests - No manual intervention required - Still safe: always rebuilds when user-facing code changes - Clear feedback about what was built/skipped 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces a helper function (`get_mvn_command`) in `bin/compile` to dynamically select the appropriate Maven wrapper (`mvnw` or `mvnw.cmd`) based on the operating system. Updated all hardcoded `mvn` invocations to use this function for improved platform compatibility. Includes minor formatting fixes.
This commit removes the Maven-specific setup, validation, and related code from the `tornadovm-installer` script, as the build system now relies solely on Maven wrapper (`mvnw`). This simplifies dependency management and avoids redundant checks for Maven installation.
- Added build optimizations to `.mvn/maven.config` for faster feedback: - Skip Javadoc generation (`-Dmaven.javadoc.skip=true`) - Enable fail-fast on module failures (`-Dfailfast=true`) - Show timestamps in build logs (`-Dorg.slf4j.simpleLogger.showDateTime=true`) - Updated `bin/compile` to respect `.mvn/maven.config` defaults for parallel builds and cleaned
- Added a new section detailing `.mvn/maven.config` settings: - Parallel builds, colored output, default profiles, skipping Javadoc, fail-fast, and build timestamps. - Included examples to override defaults for specific builds.
This commit removes the Maven Build Cache extension and its configuration files (`extensions.xml` and `maven-build-cache-config.xml`). The build system no longer relies on caching for incremental builds.
- Adjust comments in `.mvn/maven.config` for better readability. - Remove redundant and outdated configurations (e.g., `failfast`). - Streamline documentation of build flags to focus on critical optimizations.
- Update `clean_dist_for_incremental()` to handle persistent `dist/` directories for incremental builds, ensuring a fresh start for the assembly process. - Adjust Maven clean behavior to run by default unless the `--incremental` flag is specified, aligning with master branch behavior. - Add explicit comments for better clarity on distribution directory cleaning and build process logic.
- Add guidance on faster incremental builds for active development. - Outline scenarios for using or avoiding incremental builds. - Provide example commands for both clean and incremental workflows.
- Eliminated the `exec-maven-plugin` configuration in `pom.xml` as backend file creation logic is now managed by `bin/compile`'s `post_installation_actions()`. - Updated `post_installation_actions()` to handle backend file generation using actual backend names (`selected_backends`) instead of Maven profile names (`backend_profiles`). - Adjusted `bin/compile` calls to reflect the new `post_installation_actions()` signature.
- Eliminated all references to `commons-math3` as it is no longer required. - Removed associated exclusions to simplify dependency management.
- Removed `--incremental` and `--force-assembly` flags from `bin/compile` and related logic tied to incremental builds. - Deleted Makefile targets: `incremental` and `incremental-with-sdk`. - Simplified Maven clean logic and removed module-level build optimization. - Eliminated unused functions like `get_changed_modules()` and `should_rebuild_assembly()`.
- Deleted instructions and examples related to incremental builds since the functionality has been removed.
- Extract repeated Windows OS check into a new helper function `is_win_or_bat()` for reusability and clarity. - Replace all instances of inline Windows detection with calls to `is_win_or_bat()`, streamlining the code.
…/compile` for consistency
…roved readability in `bin/compile`
…tornado-argfile.py` - Replaced the Bash script with an equivalent Python script for better maintainability and extensibility. - Updated `bin/compile` to call the new Python script.
- Added helper functions `get_export_list_paths()` and enhanced docstrings across the script. - Replaced hardcoded backend paths with dynamically generated paths. - Improved handling of export list files by centralizing path management. - Switched to consistent string formatting and variable naming.
Introduces a new Metal driver for executing TornadoVM workloads on Apple GPUs. The implementation is largely based on the existing OpenCL backend, adapted for Apple's Metal Shading Language (MSL) and Metal API. MSL code generation (MetalCodeGen, MetalNodeLIRBuilder, assembly nodes) Graal IR phases (parallel scheduling, task specialization, intrinsics, reductions) Memory management for all primitive array types and MemorySegment Kernel schedulers Runtime device context, event pool, and kernel dispatch Virtual device support for offline testing JNI native library (tornado-metal) MetalTokens for Metal-specific constants TornadoBackends.METAL enum entry Metal options in TornadoOptions Few attempts to fix ConcurrentModificationException in TornadoNativeTypeElimination
…matrix-vector multiplication
[JDK21] Add
[JDK21] Add Apple Metal backend for TornadoVM to run on Apple SiliconApple Metal backend to run natively on Apple Silicon
…rivers/metal/graal/MetalStamp.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…rivers/metal/enums/MetalBuildStatus.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…rivers/metal/graal/compiler/MetalBarrierSetLIRGenerator.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Thanos Stratikopoulos <34061419+stratika@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters