在内存中编译Webpack,但解析为磁盘上的node_modules

JavaScript Webpack

小宇宙

2020-04-03

我正在尝试使用网络包来编译内存中有效的JavaScript代码的字符串。我正在使用此处概述的内存fs:https : //webpack.github.io/docs/node.js-api.html#compile-to-memory

因此,我采用了一个包含原始javascript的字符串,将其写入内存fs,然后Web pack解析为该入口点。但是编译在第一个require语句上失败,大概是因为它无法在node_modules的真实fs中查找。

关于如何完成此操作的任何想法?

import webpack from 'webpack';
import MemoryFS from 'memory-fs';
import thenify from 'thenify';

function* compile(code) {
    const fs = new MemoryFS();
    fs.writeFileSync('/file.js', code);
    const compiler = webpack({
        entry: { file: '/file.js' },
        output: {
            path: '/build',
            filename: '[name].js'
        },
        module: {
            loaders: [
                { test: /\.json$/, loader: 'json' }
            ],  
        }
    });
    compiler.run = thenify(compiler.run);

    compiler.inputFileSystem = fs;
    compiler.resolvers.normal.fileSystem = fs; //this is needed for memfs
    compiler.outputFileSystem = fs;
    const stats = yield compiler.run();
    //retrieve the output of the compilation
    const res = stats.compilation.assets['file.js'].source();
    return res;
}

用法

var code = "var _ = require('underscore'); console.log(_);";
var bundle = yield compile(code); //should be a bundle containing the underscore source.

The error is

ModuleNotFoundError: Module not found: Error: Cannot resolve module underscore in /

This question indicates that others have tried the same thing: https://github.com/webpack/webpack/issues/1562. there's a gist referenced at https://gist.github.com/DatenMetzgerX/2a96ebf287b4311f4c18 that I believe was intended to do what I'm hoping to accomplish, but in it's current form I don't see how. It assigns an instance of MemoryFs to all of the resolvers. I've tried assigning node's fs module, but no dice.

So in short, I'm trying to set an entry point to an in memory string of raw javascript, but still have require and import statements resolved to node_modules on disk.

UPDATE

I've been able to get the result I'm looking for but it's not pretty. I'm basically overriding the implementation of #stat and #readFile in MemoryFS to check the real filesystem if it gets any request for a file that doesn't exist in memory. I could clean this up a bit by subclassing MemoryFS instead of swapping method implementations at runtime, but the idea would still be the same.

Working solution

import webpack from 'webpack';
import JsonLoader from 'json-loader';
import MemoryFS from 'memory-fs';
import UglifyJS from "uglify-js";
import thenify from 'thenify';
import path from 'path';
import fs from 'fs';
import root from 'app-root-path';
/*
* Provide webpack with an instance of MemoryFS for
* in-memory compilation. We're currently overriding
* #stat and #readFile. Webpack will ask MemoryFS for the 
* entry file, which it will find successfully. However, 
* all dependencies are on the real filesystem, so any require 
* or import statements will fail. When that happens, our wrapper 
* functions will then check fs for the requested file. 
*/
const memFs = new MemoryFS();
const statOrig = memFs.stat.bind(memFs);
const readFileOrig = memFs.readFile.bind(memFs);
memFs.stat = function (_path, cb) {
    statOrig(_path, function(err, result) {
        if (err) {
            return fs.stat(_path, cb);
        } else {
            return cb(err, result);
        }
    });
};
memFs.readFile = function (path, cb) {
    readFileOrig(path, function (err, result) {
        if (err) {
            return fs.readFile(path, cb);
        } else {
            return cb(err, result);
        }
    });
};


export default function* compile(code) {
    // Setup webpack 
    //create a directory structure in MemoryFS that matches
    //the real filesystem
    const rootDir = root.toString();
    //write code snippet to memoryfs
    const outputName = `file.js`;
    const entry = path.join(rootDir, outputName);
    const rootExists = memFs.existsSync(rootDir);
    if (!rootExists) {
        memFs.mkdirpSync(rootDir);
    }
    memFs.writeFileSync(entry, code);
    //point webpack to memoryfs for the entry file
    const compiler = webpack({
        entry: entry,
        output: {
            filename: outputName
        },
        module: {
            loaders: [
                { test: /\.json$/, loader: 'json' }
            ]
        }
    });
    compiler.run = thenify(compiler.run);

    //direct webpack to use memoryfs for file input
    compiler.inputFileSystem = memFs;
    compiler.resolvers.normal.fileSystem = memFs;

    //direct webpack to output to memoryfs rather than to disk
    compiler.outputFileSystem = memFs;
    const stats = yield compiler.run();
    //remove entry from memory. we're done with it
    memFs.unlinkSync(entry);
    const errors = stats.compilation.errors;
    if (errors && errors.length > 0) {
        //if there are errors, throw the first one
        throw errors[0];
    }
    //retrieve the output of the compilation
    const res = stats.compilation.assets[outputName].source(); 
    return res;
}

Usage

var code = "var _ = require('underscore'); console.log(_);";
var bundle = yield compile(code); //is a valid js bundle containing the underscore source and a log statement logging _.

If there's not a better way, then I'll definitely encapsulate this into a subclass of MemoryFS, but I'm hoping there's a more sane way to accomplish this with Webpack's api.

第3926篇《在内存中编译Webpack,但解析为磁盘上的node_modules》来自Winter(https://github.com/aiyld/aiyld.github.io)的站点

1个回答
Tom凯 2020.04.03

代替memory-fs,unionfs / memfs / linkfs的组合应该会有所帮助。

问题类别

JavaScript Ckeditor Python Webpack TypeScript Vue.js React.js ExpressJS KoaJS CSS Node.js HTML Django 单元测试 PHP Asp.net jQuery Bootstrap IOS Android