RATIONALE
To make it possible to dynamically load engines (and other plugins)
on Windows.  (Currently the code is built in the wrong way.)

SUMMARY
Currently it is not possible to build a dynamically loadable engine as a 
separate component (.DLL) on Windows. This task is to track the effort of making
it work on Windows.

Note added by Trudy Pelzer, 2006-12-22:
Remarks from WL#3364, cancelled as a duplicate task:
"As part of WL#3201, storage engines have been made "plugable", except on 
Windows. This Worklog is to make plugable storage engine work on Windows."

REFERENCES
WL#3859 Plug-in Service API

• WL#2936: Falcon & MySQL plugin interface: server variables
• WL#3201: Configure support for server plugins

Storage engines have code dependencies within the MySQL server that need to be 

resolved at load time.  Though it is not normally or commonly used, Windows 

executables can export symbols that are used in loadable modules such as dlls.



The method is surprisingly similar to what is done on Linux.  Functions and 

data are marked as exported out of the server and then imported using a import 

library into the storage engine at compile time.  Then, when the engine is 

loaded into the server at runtime, the entry points are mapped up using the 

normal loading process.



===The main problem===

The difference between Windows and Unix  is that Microsoft linker needs 

the list of exports, while Unix linkers typically export every global symbol 

from the binary.  

Storage engines traditionally use mysql functions and data in somewhat random 

manner, and thus we'll take  a practical approach and export all global symbols 

from server objects and selected libraries. We will implement a script to 

produce list of all global symbols from static libraries and objects.





===Importing data===

Microsoft linker uses special decorations when exporting data (i.e variables) 

from dynamic libraries. Users must use compler extension __declspec(dllimport) 

to access the data. For this purpose, a MYSQL_PLUGIN_IMPORT macro will be 

created and all data currently used by plugins will be marked as 

MYSQL_PLUGIN_IMPORTed.



===Future===

This is a stop gap solution until WL#3859 (Server API) is implemented.

Quirks( pre-link step, scripting and MYSQL_PLUGIN_IMPORT) can be removed after 

this WL is finished.



===Known limitations===

- Dynamic storage engine plugin support will not be available within embedded 

server library. DLLs we produce this way have dependency on mysqld.exe, and 

they will not load if current process is not mysqld.exe.

- /GL compiler option (that gives best possible optimization) will not work.

Objects produced with /GL are not in the COFF format and cannot be parsed by 

dumpbin.





==Support for plugins in build tools : win\configure.js and CMake==

Command line switches for win\configure.js will be made compatible to Unix with 

respect to plugins (refer to 

http://forge.mysql.com/wiki/MySQL_Internals_Support_for_Plug-Ins for details). 



Specifically --with-plugin-PLUGIN, --without-plugin-PLUGIN and --with-plugins=

[plugin-list or group] will be supported.



Also, win\configure.js will continue to support existing 

WITH_<PLUGIN>_STORAGE_ENGINE syntax. This option has the effect as --with-

plugin-PLUGIN and means that PLUGIN should be statically compiled into the 

server, if it supports static build. If plugin does not support static build 

(currently example storage engine only), WITH_<PLUGIN>_STORAGE_ENGINE and --

with-plugin-PLUGIN will have no effect.



Inclusion/exclusion logic is the same as on Unix, that is :

For any plugin that is not explicitly or implicitly (as a member of a selected 

group) selected or disabled, it is selected to be built dynamically if it 

supports dynamic build, and is disabled if it does not support dynamic build. 

If no plugin options are given, default group is selected. 



CMake will continue to support WITH_<PLUGIN>_STORAGE_ENGINE and now will also 

understand WITHOUT_<PLUGIN>_STORAGE_ENGINE. CMake will not support grouping 

internally, groups must be resolved within configure.js.

==How plugin-specific configure parameters are passed to Cmake==



win\configure.js will include logic to translate --with-plugins/--without-

plugins to CMake's WITH_<PLUGIN>_STORAGE_ENGINE/WITHOUT_<PLUGIN>_STORAGE_ENGINE



This will also handle grouping, so if someone passes --with-plugins=max, it 

with be expanded to a big list of WITH_<PLUGIN>_STORAGE_ENGINE 



==Parsing plugin definition files <source-root>\storage\*\plug.in==



Cmake will parse plugin definition file in storage engine directory plug.in to 

find out whether plugin is mandatory, can be built statically or dynamically.



If will decide, based n WITH_<PLUGIN>_STORAGE_ENGINE parameter and plugin 

defintion whether to build and if yes, how to build.



== Cmake macro MYSQL_STORAGE_ENGINE==



CMakeLists.txt for every storage engine will use special macro 

MYSQL_STORAGE_ENGINE(<PLUGIN>), defined in 

<source-root>\storage\mysql_storage_engine.cmake. 



This macro basically only needs <PLUGIN>_SOURCES to be defined,and does the 

rest (making static library or DLL, setting include path) automagically.



=== Exporting functions and data from mysqld ===



1.  Create a pre-link step that will run a script file after compilation of the 

server files but before the link step.

2.  This script is a jscript file that runs the dumpbin utility on all server 

object files and some of the libs that get linked in statically. The script 

needs to tell data from function and export data as "symbol_name DATA"  

3.  Step 2 above produces a module definition file that is used as input to the 

link phase for mysqld.

4.  The linker will now produce an import library for the server.  This import 

library is then used as input for all pluggable storage engines.



=== How to determine which variables needs to be MYSQL_PLUGIN_EXPORTed ==



1. Configure server with all plugins as dynamic - win\configure.js without 

parameters will do that. 

2. Try to make. 

3. If plugin DLL complains about unresolved symbol, find variable declaration, 

add MYSQL_PLUGIN_IMPORT to it. Goto step 2.



===Quirks and workarounds==

Ideally, all plug.in. parsing would be inside CMakeLists.txt. We do not do it 

now, and parse groups from within configure.js. This is a workaround for CMake 

regex limitations (own syntax, absence of non-greedy matches), that makes 

parsing anything non-trivial quite a complicated task.

You must be logged in to tag this worklog