A New Chart Making Program. Update 21 Feb 2018

[SteveM]

That sounds like a great idea. I'd love to be able to tweak the aggressiveness of the dither as I don't think that perfect error correction should be the goal with pattern making and hate it when I see a checkerboard effect like this:
XOXOXOXO
OXOXOXOX
XOXOXOXO

[richardandtracy]

I seem to have got a softer dither using a custom dither definition window. I fear it is even slower than the other dither methods, but possibly the wait is worth it.
Original Sierra 2-4a conversion on the left and revised custom version on the right


The custom dither definition window is below:

In the Sierra 2-4a dither the divisor is 4, and when it's increased to 5, the dither seems a little softer, as seen in the image on the right above. It should be noted that allowing people the ability to fiddle with another 13 variables means there is a HUGE increase in chance of them getting a poor conversion.

Regards,

Richard.

[SteveM]

I love it! As I lean back from my computer the softer image with custom dithering blends sooner than the image on the left. Between your inclusion of the Anchor blues and softer dithering this is truly a great result not obtainable with commercial software products, no matter how long the wait. By the way, divide by 4 was likely optomised into a right shift of two by the compiler while divide of 5 has no such shortcut.

[richardandtracy]

I'm sure you are right about the divide by 4. However, I have not programmed any dithering that way - the >> and << commands are 'highly advanced' for me, and have only ever used them in the RGB colour to R,G,B component conversion (where it did make a noticeable performance boost).

Not sure when I can release this - writing the help file will be a nightmare, and it's so complex I need to have something written down to remind myself (if nothing else). I hope Friday.

Regards,

Richard.

[richardandtracy]

Just uploaded the revised download. Now it's 4.29Mb - growing rather larger than I want, to be honest, but there doesn't seem to be a lot I can do about it.

Regards,

Richard.

[SteveM]

I'm just back from holiday where I wasn't able to test the new program as the windows laptop that I had was having a terrible clash between the VPN and firewall software that couldn't be resolved till I was back at work today. But I was able to play with it a little over the weekend and am very encouraged by the dithering customization.

One interesting discovery was that Pattern Maker doesn't use the same RGB values for the Anchor blues as you do, they show them being very similar to the DMC colours. Since I don't have any floss to look at I am unable to say which program is more correct, but hopefully yours is as the blues look so much better.

[richardandtracy]

My Anchor colours have all been taken from a pdf file of a floss card (Coates Art.-Nr. 003001189061 CC-089900118 dated 06/04 [presumably June 2004]) that I found on the web a couple of years ago from the now apparently defunct crossstitchtreasures.com. If you want a copy of the file, PM me with your e-mail address & I'll send it to you. The card was one of those with real threads on it and had been photographed in one photo, ensuring that the same colour errors were present in all the colours, so even if there was a colour cast, at least it's the same for all the threads and a stitched image would then have a reverse colour cast pretty much equal & opposite to the floss list colour cast. In practice, I do not think the colour cast (if present) will be significant.

I then used a program I'd written (and is shortly to be released as part of my charting program to enable someone to create a floss list from scratch) that enabled a part of an image to be pasted into the program, then by dragging a rectangle over the image, the program averaged the colours within that rectangle, obtaining averaged RGB components. That way the darker parts of the thread in shadow were averaged against the highlighted areas of the thread to give an attempt at a 'real' colour for finished threads as opposed to a theoretical colour. It seemed to be a fairly good way of doing it that gave fairly consistent results from different parts of the same floss image. After going through all the shades on the floss card over several days, I saved the generated file as a comma separated variable file (.csv). Finally - and I have no plans to replicate this in my charting program - I stretched the RGB ranges to cover the whole spectrum. The RGB component values obtained from the floss card ranged from 3 to 247 and were pretty much the same in the Red, Green & Blue components. Using a spreadsheet, I applied the same factors to all the Red colours to extend the range to 0-255, and did a similar thing to the Green & Blue components. The fact that slightly different factors were used in the Red, Green & Blue components will have altered the hue very slightly, but having seen a 'before' & 'after' colour I was very hard pressed to tell a difference. Having done this, I then regenerated the .csv thread definition, and those are the colours in my program's Anchor range. They are very, very unlikely to agree with anyone else's figures simply due to the fact that they are original data I have generated with reference to a particular floss card rather than anyone else's theoretical colours.

AmandaG did send me some CMYK data regarding Anchor colours, but I was unable to find a definition of CMYK->RGB conversion that seemed to give colours that came close to the floss card I had, and with great reluctance I had to fall back on my own resources. There seem to be a number of different CMYK->RGB conversion formulae, which does concern me as to whether they are even compatible ways of describing colour that one can convert between. I suppose they are, as many people seem to do it, so once again I'm simply showing the depth of my ignorance.

Regards,

Richard.

[richardandtracy]

Just managed to upload the revised version with the ability to generate thread definition files from scratch or by copying/editing existing definition files. The help file is revised too.

Regards,

Richard.

[SteveM]

Richard,

I thought I'd post my George Washington cloak pattern examples here as the other thread is more about confidence in the SQ colors than comparing it to what other programs can generate.

For reference, the original artwork found on Wikimedia and scaled to the same size as the pattern:


A mediocre example of PM and 151 solid DMC colors with dithering:


BlendThreads with 583 colors, using blended threads, and no dithering:


BlendThreads with 659 colors, using blended threads, and custom dithering with a divisor of 10:


While I am focusing on the cloak as an example, your program hands down has made the absolute best sky on the smaller 538x345 size that I have been working with. I have also made no attempt to reduce the number of colors used through merging or hand editing which could easily reduce the color count by 10% or more.

What I have learned also is that all images benefit greatly in detail and color correctness when the image size is increased, say to 999 on the longest dimension. When the entire image is sized to 999x640 PM produces a good result with up to 175 solid colors but dithers with some purple that I find a bit distracting and could be modified. The BlendThreads program at 999x640 with blending and 559 colors (before reduction) and with a custom soft dither is just stunning and would only need minor touch up. I took a quick look and "only" 400 colors are used with greater that 10 stitches each so a large reduction in colors used could be achieved with merging.

PM 999x640:


BlendThreads 999x640


PM seems to handle the greens better, but in every other regard the BlendThreads pattern looks much better with colors that just pop off the page.

Original at 999x640 for reference:

[richardandtracy]

Ye Gods... That is stressing the program well beyond anything I could possibly have imagined when I started writing it. When I started in 2011 it was just intended to cover a deficiency in Ryijy because at that time Ryijy would not print on more than one page..
The fact it can do a conversion of any sort on picture of that size boggles my mind. And to be no worse than a commercial program. You have made my day... I will try to think of a more coherent response once I have stopped grinning.

Do you think it might be worth while adding an auto merge in the colour conversion routine for stitch numbers below a certain user specified threshold? That could reduce the numbers a fair without significant user effort. There is a risk of missing a glint in someone's eye though, which is why I think it should be possible not to specify it too.

One thought - I may need to programme in more symbols, and I thought 600 would be many more than anyone would ever need.

Regards

Richard.

[SteveM]

I'm glad you can see your program at work on some of these larger pieces of art. This isn't the largest one I've thrown at it, but it certainly is up there. I believe the average colors per 100 count square was only 11 with the worst at 83.

An optional auto merge would be useful. Even if all it targeted was the colors with just 1 or 2 stitches that would save a lot of manual work. Trying to find a single highlighted stitch on these large patterns can leave nose prints on your computer screen. I have heard of the glint in the eye stitch so options are always good to have.

I don't think >600 symbols is necessary. To be honest an acceptable result is often better with fewer colors, I was just trying to see what could happen if I opened it up and let it go crazy.

One small suggestion I have to add is that the floss usage charts are based on assumptions of how certain fabric types are used. For example, at 22 and above thread count it assumes stitching 1 over 1 instead of letting me stay 2 over 1. I'd rather be able to manually specify thread count, stitch count, and stitch type. Minor detail of course, but it would make the pattern printing more flexible.

[richardandtracy]

I must admit that for some while I have felt a little uncomfortable about the assumptions of how people are going to stitch it, but felt I'd covered most common usage. As you have mentioned it too, I'll do something about it.

I think I'll do it as another series of dropdown combo-boxes. There is already one for the count. I'll see if I can turn it into a sentence with a combo box for each underlined section in the sentence below:
'The image will be stitched using 2 threads over 1 hole(s), on 32 count evenweave with a full cross'

The options will be
For threads: 1,2,3,4,5 or 6
For holes: 1 or 2
For the count, 6,7,8,9,10,11,12,13,14,15,16,18,20,22,25,28,30, 32,36 or 40
For the fabric type it will be 'aida', 'evenweave', 'linen' or 'fabric'
For the stitch type it will be 'a full cross' or 'a half cross (tent stitch)'

Regards,

Richard.

[richardandtracy]

I have incorporated the changes I talked about earlier and have uploaded a new version of the program. The user selects the number of threads, number of skipped holes, count, fabric type and stitch type. The effects are quite extensive - stitching 2 over 1 had a fabric size and thread quantity half that of 2 over 2. This is a little cutting from the program main window:


In the conversion window threads with between 1 and 25 stitches can be automatically merged with their nearest matching colour if so desired. I tried a threshold of 3 stitches on a 750 x 450 image with dithering. The thread number dropped from 650 to 410, at the expense of about 3 seconds conversion time. This is the area on the thread conversion window:



I may have speeded up the opening of an image too. For some reason I seem to have had two calls to the thread array allocation method about 10 lines of code apart. On big images, this routine takes quite a few seconds to go through all the stuff it needs to do, and doubling up on it was slightly unhelpful.

As a last word, the source code is now available from the main page too http://www.chestnutpens.co.uk/misc/crossstitchplan.html. The zip file of the code is 473kb. I shall not keep it up to date, but it may be of interest to someone.

Regards,

Richard.

[richardandtracy]

I have updated the program with the following extra features:

1. The user can add page margins in addition to the unprintable margin around each page of the chart.
   
2. Blends now sort so the smaller number comes before the bigger, and when sorting by floss number, the blend list is numerically sorted rather than string sorted (means that '996+3017' comes before '3489+3614').
3. You can choose to print the Detailed Key in black & white, rather than printing being in colour only.

Anyway, it's possible to download now from the link in my signature. If you have any older versions of the program, the blend definition files (*Blend.csv) in the program directory need to be overwritten from the 4.5Mb zip file for the sorting to work properly.

Hope this improves things a little,

Regards,

Richard.

[Allyn]

Thank you!

Yippee! Downloading right now.

[SteveM]

I've been playing around with this latest update and really like the automatic merge during image import and the ability to specify my exact stitch type and fabric count. Auto merging with the default of 3 reduces the colour usage by about 10% which is what I was estimating based on output from previous versions of the program. I am really delighted with what you've been able to accomplish and am working to find a smaller design that I can actually use it on and stitch up.

I decided to make an apples to apples comparison to Pattern Maker and loaded up my sailing ship test image into both and used an Anchor solid colour palette with 218 colours in by both programs. At first glance the results looked quite similar, but then I could see that the PM sky was more baby blue than a medium blue and the usual PM problem had reared its ugly head - dark areas had been turned into black. The dark detail can be brought back to life by increasing the brightness, but that causes the sky and clouds to wash out. Thanks to your custom dither I am also able to match the softer PM dithering. PM is fast and well refined compared to your program, but I think we'd all agree that the quality of the pattern is what matters most and I give your program the edge over PM even before blended threads or mixed DMC+Anchor palettes are turned on.

I thought I had found a solution to being able to use my Macbook by using Remote Desktop to the windows computer, but something about that causes the image import window to not close when I click "OK" and it simply returns to all of the default import settings. This isn't something to debug, I'll just have to sit at the windows computer when doing image importing.

[richardandtracy]

Must admit it's a problem I couldn't debug anyway. I have no access to any Apple sstuff at all... Oh, apart from the two I eat every lunchtime.

Glad to hear the program is working well for you. Not sure how to make it faster, without becoming a computer professional. Not sure I have time for another degree along with everything else.

Regards,

Richard.

[SteveM]

No worries, stuff like that would be nearly impossible to debug even if you had all of the tools needed to do it and I can simply plop down in front of the windows machine as needed.

I've been playing around a bit more with reducing the number of colours used by being more aggressive with the automatic merging and allowing more colour space divisions. I've got a pattern that I made with DMC + Anchor blues with blending and Sierra 2-4a dithering and instead of only 10 or 11 colour space divisions I can use 14 because auto merging is an aggressive 20. The end result is the most pleasing yet though it is using 357 colours to do it.

I have two suggestions for feature additions. The first is a shopping list of skeins based on the selected stitch and fabric type and the other to automatically assigning dark symbols to dark colours and light symbols to light colours.

[richardandtracy]

Steve,

The 'Skein Shopping List' is already there in the 'Detailed Key'.
There are two versions of the Detailed Key.

1. When there are no blends, there is a column after the number of stitches that says 'Approx Number of Skeins', there is an integer and another smaller number in brackets rounded up to the nearest 0.1. The integer is the number of skeins you need to buy if you don't have any of the colour at all, and the smaller number is the proportional skein length you actually need, just so you can check your existing stash.
2. When there are blends, the 'Approx Number of Skeins' column is visible in the second half of the Detailed Key where the individual DMC numbers are specified.

The length is calculated based on type of stitch, fabric count, number of threads, number of holes skipped etc just as the chart is printed. Which is why I get the user to fill in the sentence 'The picture will be stitched using 1/2/3/4/5/6 threads, skipping 0/1 hole(s) on 14/18/24etc count (tpi) fabric/aida/evenweave/linen with a full cross/half cross.' as the sentence contains all the additional info needed for this calculation.

I shall see whether it's possible to make the program automatically allocate symbols immediately after conversion. At the moment the allocation is done in a separate window, which (in accordance to C++ programming practice) is invisible to the conversion window. I would also have to make it a voluntary decision with a checkbox - the reason; I programmed the symbols I like and am used to in the first 100 symbols and don't want them automatically taken away from me on an involuntary basis. At the moment you can actively decide to have darker symbols for darker stitches after conversion by selecting menu option 'Edit|Set Chart Symbols for This Chart Only' and then pressing the 'Sort' button.

Regards,

Richard.

Edited to add:
I could possibly kludge the symbol brightness by putting a checkbox in the conversion window, then in the main program setting a bool value, finally invisibly create the set symbol window, and program in the pressing of the 'Sort' button. Bit of a kludge & not very elegant. May have to do it another way.

[SteveM]

Well don't I feel dumb. It is even spelled out in the documentation so I have no excuse for my ignorance. The symbol sorting worked well and in my test PDF printout the general shapes of the objects were visible in monochrome which I like and the skein info is invaluable considering this was a blended pattern. I don't think you need to do anything with the symbol sorting, as long as there is a way like there is now I'm happy.