Talk:MySQL Build Farm Initiative
[edit] Votes/Comments
1.1 Until you get more then 1 server a day in registration, i'm sure a manual process of emailing some criteria would surfice. More important to get the pull/push working. --Ronald
I suggest a simple form. Set up the form NOW, so folks have a consistent interface to go to. Have the form just e-mail for now, later it can go into a db with other stats (how many builds/tests have been run, status, etc) if you want. Example below [brackets denotes Jay's example]:
[Jay Pipes] Name (first + last, don't think it needs to be 2 fields, but can be) [elided 'cause this is a webpage] Email [Dell] Vendor [GX620] Machine [2] # Procs/Cores [Pentium 4] Proc/Core type (separate out to vendor and version?) [3.2 Ghz] Proc/Core speed [] Proc/Core cache size [1 Gb] RAM [] swap space [Ubuntu] OS [6.06 LTS] OS Version (not sure what LTS is....) [2.6.15] Linux Kernel / Windows service pack [] 64 bit or 32 bit?
On the page with the form, put (also add the windows and mac os x ways to find these things): "uname -a" gives you OS, OS version, Linux Kernel (or "cat /proc/version" if uname gives you "GNU/Linux") "cat /proc/meminfo" gives you RAM and swap space "cat /proc/cpuinfo" gives you # procs/cores, proc/core type, proc/core speed, proc/core cache size ???? gives you vendor and machine -- Sheeri
See Commands to Gather Hardware Specs for my list -- Ronald
1.1.1 See the new BuildFarmInventory page to enlist your own hardware --Jpipes 15:23, 2 August 2006 (CEST)
1.2 Works for me, can we just work on the same type of page. --Ronald
2.1 Initiation for the first time, should be to download and configure (e.g. cron) a script. This script manages getting from the farm server an archive which includes both the source tar, and the compile/logging script. Indeed if an updated version of this initiation script is needed it's included in the process and can replace itself on appropriate machine. (Note: I have no idea how this works in a windows environment, cygwin perhaps?)--Ronald
Windows Installer should be able to deal with that (or some kind of Windows updater -- they exist out there) Sheeri
2.1.1 I am in the process of talking with Kristian Nielsen about the benefits of a push versus a pull architecture. With a push architecture, enlisted clients can be notified immediately of a new snapshot, versus a pull architecture which would check for new snapshots on a regular basis. The push architecture would work more smoothly with our existing pushbuild system, however, it may not be ideal for a highly distributed system like this. I'll add more here when we talk more. --Jpipes 15:23, 2 August 2006 (CEST)
2.1.1.1 Pull has the advantage to have a chance of working behind firewalls with much hassle. --Sebastian 09:16, 3 August 2006 (CEST)
2.1 Infrastructure needs to cater for environments that have multiple servers within a local network. Some level of local caching of the snapshot is required, so the major download isn't repeated.--Ronald 6 Aug 2006
2.2 With appropiate authentication (see 4.1), process should just get an availability file generated by determining the last pushed results, that contains any newer archives, and then processes these. --Ronald
3.1 The script run to compile (that's included in the pulled archive), should produce 2 logs, a script log with system,statisically and results information, and a detailed log file of all stdout, for debugging purposes), the first log being sent back to farm server, and then available as the config link in the results.--Ronald
3.1.1 This is also something I am talking with the build team about. Currently, pushbuild produces a very detailed log of the build and test process. Not sure if we want to keep the same log format. Kristian and Kent will have more input on this --Jpipes 15:23, 2 August 2006 (CEST)
3.2 If the running script to compile is controlled from the farm server (each time you get an archive), the output can simply be parsed via basic sh comments. While XML is nice, it's an overhead in the first initial simple version, KISS. YAGNI.--Ronald
Why not quoted csv, escaping quotes in the data, and then it can be easily imported to a CSV storage engine table? Why make it harder than necessary? (unless MySQL is getting an XML storage engine soon....) --Sheeri
4.1 If it's being openly displayed on a status page, encyption isn't necessary. Part of registration could be to provide a public RSA key to MySQL Forge that enables SCP access to pull/push files.--Ronald
I agree -- or, perhaps, a visible web page. It needn't have info other than the server ID. Why not have the Forge page poll a web page regularly?
5.1 -- mysqld_multi perhaps, to set up multiple servers on one machine?