Welcome to Emulationworld

Forum Index | FAQ | New User | Login | Search

*View All ThreadsShow in Flat Mode*


SubjectDone -- Multi-Machine processing for ZipMax Reply to this message
Posted byPacFan
Posted on01/30/04 11:50 AM



I have the changes complete, compiled and tested.

I will email Roman with the changed source code and readme and ini file to be updated to .51 if he approves of the changes.


Here is a description of the feature as implemented:

How can I use multiple computers to process a bunch of files without manually
separating chunks of work or having them process the same file more than once?

Zipmax version .51 introduced 3 new zipmax.ini file settings: SkipReadOnly,
SkipArchive and SetCurrentReadOnly. By setting these all to 1, you will
be able to point 2 or more machines at the same list of files from a network
share and not have to worry about separating them or duplicated processing.

How to use it:
==============
1) Mark all of your zip files NON read only and NON archive ready.

In Windows Explorer, simply highlight all the files and/or folder(s), right
click, choose properties, clear the Attributes: Read Only checkbox. Click
the Advanced button and clear the "File is ready for archiving" checkbox.
Click OK and OK.

From Dos, simply cd into the folder containing the files, and type:

attrib -r -a *.zip

You may use the /s flag to also change the attributes of files in subfolders.


2) Set the ini file settings (SkipReadOnly, SkipArchive, SetCurrentReadOnly)
to 1 and relaunch ZipMax.

3) On the first machine, drag and drop all the files/folder(s) to be
processed into ZipMax.

4) On the second (or more) machine(s), from the same folder over a network,
drag and drop the same list of files/folder(s) to be processed onto a separate
copy of ZipMax on each machine.

5) That's all. As the files are processed, those that are already being
worked on by one machine are tagged ReadOnly, and therefore will be skipped
on all other machines. Also, those files already completed by a machine will
be tagged with the Archive attribute and subsequently skipped on other machines.

By using the zipmax.ini flags, this ensures the same file is not processed
more than once by any machine, wasting time.


Special notes:
==============
Rather than dragging and dropping an entire folder (which sorts the same
on every system) you select the files and choose to drop from large to small
on the FASTEST processor system (or the main system not going over a network)
and from small to large on the slower or networked system.

This will ensure those files that are the largest can finish faster on a fast
system or less data needs to be transmitted over a network (e.g. especially
if you are using wireless).

It also reduces the chance for collision. Athough one system will skip files
already marked A or R, exact timing could cause both systems to grab the file
before changing the status to read only and both work on it, with the faster
one updating it and the slower one receiving an error when attempting to
replace the original zip (now changed by the other system). Tests show only
limited < .5% Error collision on identical ordered lists. Changing the orders
could limit potential collission to 1 total no matter how large the list is.
Please note: Even though one system reports an error (not skip), the other
system will succeed and the file will be compressed and archive bit set.

You must have a separate copy of the zipmax.exe and zipmax.ini (and all
required archives) on each machine IF you plan to write to a log file,
otherwise you can point to the same folder on the main machine)


skipreadonly = 1 or 0. Set it to 1 if you want files marked read only
(R bit set) to be skipped

skiparchive = 1 or 0. Set it to 1 if you want files marked for archive
(A bit set) to be skipped

setcurrentreadonly = 1 or 0. Set it to 1 if you want files that are being
processed to be set to read only. Note: files will revert
to non-readonly when completed even with 0 byte change


-
Entire Thread
Subject  Posted byPosted On
*ZipMax - New setting to assist in distributed processing  PacFan01/29/04 11:56 AM
.*Re: ZipMax - New setting to assist in distributed processing  PacFan01/29/04 12:11 PM
..*Re: ZipMax - New setting to assist in distributed processing  Roman01/29/04 12:40 PM
...*Re: ZipMax - New setting to assist in distributed processing  PacFan01/29/04 07:14 PM
.....Done -- Multi-Machine processing for ZipMax  PacFan01/30/04 11:50 AM
.....*Re: Done -- Multi-Machine processing for ZipMax  Roman02/02/04 04:13 AM
....*Re: ZipMax - New setting to assist in distributed processing  f205v01/30/04 04:31 AM