A New Chart Making Program. Update 21 Feb 2018

[NeedleAndFork]

How slow is 'really slow' ?

I'm used to working with photoshop on multi layer images etc where I can fully expect to sit and twiddle thumbs for a couple of minutes while waiting for things to calculate and process. Graphics are a resource hog! As long as your software doesn't keep the resources tied up even after the program is closed and done with, I think most people would tolerate waiting for things to process as long as the wait is reasonable.

I am seriously in awe of all the stuff you've put into this btw. I'm already pretty sure my next large project after my current HAED will be something I create using your software. I don't know what yet, simply because I can't choose a single image I like best. Besides, there's still a while to go, I'll change my mind and find a dozen other images I like between now and then.

Though.. I did just get an idea. I'll have to play with my camera a bit and see what I can come up with before sharing. This could be very meta.

[richardandtracy]

I have a relatively fast machine on my desk (3GHz quad core Intel I5 with 8 'virtual' cores), but due to the fact I'm not multi-threading, it's all going through a single core and it's taking 20-30 seconds on a 500 x 600 pixel image. Due to the fact I didn't put an hourglass cursor on the operation, I thought it had frozen first time I ran it.

It is possible to speed things up by using a 2x multiplication factor instead of 4x, but the reduction in quality is possible to notice. I think I may offer 3 levels of 'Subspace Sampling' to use the same jargon as everyone else. Zero, for quick & dirty WindowsGDI, 2x (ie 4 pixels) for better quality and 4x (ie 16 pixels) for best quality but slowest operation. It has to be a multiple of 2 as I'm using bitwise operations for the multiplication & division as they're quite a lot faster than integer maths.

At the moment I'm trying to draw the feature on a full bitmap in memory held at 4x size, so for a 500 x 600 pixel image the bitmap in memory would be 2000 x 2400. Then all 2000 x 2400 pixels are reduced in size. This is regardless of whether the feature is just a line from (say) 210,30 to 220,35 on the normal size bitmap. If I'm feeling clever at some point in the future (a very rare event), I'll cut this area down so there is a border around the object being considered, and then just do the subspace sampling on a much smaller area, which will speed it up lots - but the thought involved behind the programming will be intensive.

I am very careful to make sure that every object I create is deleted after use in the program. I once did a memory leak check with my 3D modeller, and after handling 500k elements, rotating them, scaling the model, deleting some and exiting, the memory available to windows was the same to the byte as it was before opening the program. While in investigatory mood at the time, I did the same with Word 2003, and found that after nothing more than opening and closing the program there was 170K less memory available every time I did it.


I am intrigued by what a 'Meta' project could be. The term conjures up ideas of stitching an image of an image being converted for stitching, but I can't imagine that it would be a great picture.

Regards,

Richard

[NeedleAndFork]

I am very careful to make sure that every object I create is deleted after use in the program. I once did a memory leak check with my 3D modeller, and after handling 500k elements, rotating them, scaling the model, deleting some and exiting, the memory available to windows was the same to the byte as it was before opening the program. While in investigatory mood at the time, I did the same with Word 2003, and found that after nothing more than opening and closing the program there was 170K less memory available every time I did it.

That's impressive. I wish everyone else that wrote software was as careful as you are. Web browsers are especially horrible, each time I open a new tab, that's another ding that never completely goes away until I reboot. And I often do stuff that involves opening(and closing) hundreds of tabs within an hours span.

I am intrigued by what a 'Meta' project could be. The term conjures up ideas of stitching an image of an image being converted for stitching, but I can't imagine that it would be a great picture.

Actually, you're onto my idea there. I've always been intrigued by pieces that are images of people invovled in fiber/needle crafts. Scarlet Quince has one called I'm Half Sick Of Shadows that always amuses me because in addition to the actual meaning of the line in the poem, she looks like she's fedup of all the shadows and confetti stitching in the piece she's working on. There's another pattern that I don't remember who it's by that is of a woman with a drop spindle and some wool that she's working on. The concept of creating these images in needlework is one that appeals to me, but neither image itself screams to to me to spend the time it would take to stitch it. So.. I'll have to create my own image. I was thinking about the photos I take sometimes - extreme macro shots where you can practically see individual fibers within a strand of cross stitch thread. It might be interesting to take a picture like that of just a small area of stitching in progress, maybe with scissors and a thimble resting on the work and a small amount of the fabric visible, and turn that into a pattern. It would mean that each 'stitch' in the photograph would be large enough to be made up of many many stitches and colors to show off the shadows formed by the thread. Other ideas I have are of kniting - either lace or a heavily cabled pattern in progress on some of my rainbow wood needles .. drop spindles (I have some amazingly beautiful wooden carved ones as well as metal ones made from Nepalese mandalas), an image of my spinning wheel in motion.. etc. Basically the things that a crafter sees up close while working on their craft.I like to collect and use beautiful tools for the things I make and I think that a series of images like that would make for incredible artwork to hang in my studio space, assuming I ever get it cleaned up and not treat it like a dumping ground for junk!

I've got allll sorts of ideas now!

[richardandtracy]

Never thought of making Knitting needles from pen blank material...! Those diagonally dyed woods are often used for pen blanks (http://www.theturnersworkshop.co.uk/sho ... lanks.html)

That is one heck of an idea for a stitching project. Could be quite fun. Reminds me of an image I once saw on renderosity.com of a 3D figure extracting herself from an easel with a 2D sketch of her on it. It was done with the 'Poser' 3D figure animation package in the late 1990's. Can't find it any more.

Regards,

Richard

[NeedleAndFork]

Never thought of making Knitting needles from pen blank material...! Those diagonally dyed woods are often used for pen blanks (http://www.theturnersworkshop.co.uk/sho ... lanks.html)

Ohh.. pretty colors. Knitting needles can be made of just about anything as long as the material is strong enough and can be polished to a smooth enough surface with a satisfactory tip. I have a full set of them in the Harmoney wood (that's what the company sells them as), as well as their nickel plated set. I also have assorted others in bamboo (which is nice and grippy even when polished to a shine to use with slippery threads like silk), some made of casein (milk protiens!!) some ancient tortoiseshell needles, various other plastics and metals, including a gorgeous handmade set made of bronze

As for drop spindles - the ones I linked you to are just the tip of the iceberg. There's a company that specializes in making them from rare woods and other interesting materials. I don't own, but I've tested out one made from a sharks tooth even. As you can imagine, if you love unusual or beautiful tools to work with, knitting and spinning can get very expensive very fast. And we haven't even gotten into the fibers and yarn yet. Nevermind silk or cashmere, there's quivit, camel, and yak these days. In comparison the tools for cross stitching are kinda.. dull! I mean qsnap frames are glorified PVC piping!

I played around a bit with the software last night with an image I found off the web. I'm going to have fun with this, I can already tell!

[richardandtracy]

Just to let you know of current progress. Since starting asking these questions I have done about 40 hours of pretty intensive programming. I have also learnt a great deal about graphics programming!

About 70% of the required undo commands have been incorporated.
I have implemented text rotation, at any angle of 0 to 359.9 degrees by text entry, step 0.1 degrees and anti-aliasing. The anti-alias colours are.. Interesting. I am not sure anyone will want to incorporate anti-aliasing in small text sizes.
For anti-aliasing you can decide whether to use blended colours or not, and also the number of colour space divisions.
I have implemented an editor for the 'Pencil', 'Erase' and 'Block' objects. When you edit the object the screen blanks out to a grey/white checkerboard. The left mouse button then adds a stitch and the right one removes it & makes the object transparent for that stitch. This means that initially rectangular blocks can be cropped into more organic shapes if necessary.
The lines can be anti-aliased (doesn't work as well as I hoped, will have to do some checks on this). Then the start and end points can be edited. Line widths can be changed - range 1, 2 & 4 stitches. Lines can be regenerated with the current anti-alias settings if the previous ones are not pleasant.
The flood fill can be set to fill either the pixels to left, right top & bottom of the one considered or all those around the pixel being considered (this means it's possible to flood fill a diagonal dither). The flood fill can be moved once created, and re-generated in its new position if required, filling the new colour it's sitting on top of.
At the moment rectangles & ellipses can be copied & moved only.
Erase objects update when they are moved, automatically showing the revised background chart image when it's moved.
Blocks can be linked, edited, links broken, moved & copied. When one block is edited, all linked blocks change.
The background chart image can be hidden. Anti aliasing of lines, text etc is unaffected by whether the chart image is displayed, it will always be taken into account regardless.
All objects can be selected, shown or hidden. All objects can be moved backwards or forwards through the display order.

Current code length is about 9600 lines - or about 150 pages if printed out, and it has to be character perfect or it doesn't compile.

I intend to implement rotation and mirroring for all objects using mouse picks. For blocks I want to implement a linear array for 2 dimensional patterns. I am going to make it possible to change line widths for rectangles & ellipses. Anti-aliasing of rectangles at angles of 0, 90, 180 and 270 degrees makes the rectangle look awful, so I'm going to prevent anti-aliasing at those angles. I will implement anti-aliasing for ellipses.
Once that is done I'll make an interim release.

I also want to add poly lines and filled polygons. These will be similar to lines in the way they work, but will come after the interim release.

I hope this gives an idea of what I'm up to - just because I've been quiet about it doesn't mean no progress.

Regards,

Richard.

[cairee]

wow

you have been hard at work on this for sure! Cant wait to see the next release

[richardandtracy]

Would anyone be desperately upset if there was no option to mirror text?

I have found a way to get windows to rotate text, but mirroring each character does not seem to be part of the font drawing repertoire. I don't think changing the system to right to left text would be a wise idea, as it would affect the whole PC. It is possible to copy across the mirrored pixels, but almost impossible to ensure that anti-aliasing is done correctly on a multi-coloured background.

I don't think mirroring text would be particularly useful in practice, anyway. Stitched objects are opaque, which means that the only way to see mirrored text the correct way round would be to look at it in a mirror - and how often does that happen?

Unless someone kicks up, I'll give mirrored text a miss.

The other compromise I have made is that you can choose only between horizontal and vertical mirror lines. Doing angled mirrors for pixel based things like points, flood fills, erase objects and blocks causes a lot of grief due to rounding of integer values of pixel locations - it ends up with 'holes' in filled areas because rounding to integer values places coloured pixels on top of each other and a hole next to the overlap. There is nothing I can do about this - it's a feature of integer based maths.

Regards,

Richard.

[Allyn]

No kick from me. I agree on the mirrored text thing. Mirrored text brought to mind a stitched piece that a friend's mother had in her home's entryway. Looking at the stitching, it looked like a pretty spray of scrolling vines with little flowers. On the opposite wall was a mirror. When you looked in the mirror at the stitching, the scrolling vines spelled out "Welcome." I always thought that was very clever. I know that's not what you mean when you talk about mirrored text, but it made me think of that piece.

[richardandtracy]

Actually, Allyn, that was exactly the sort of thing I was thinking about. It's so rare that it sticks in your mind, and in all probability it's not going to be something that's done often.

It will be possible to do in my program, in a way. Put in the text the correct way round, define a block over the text, then mirror the block & delete/hide the original block. This will only be viable if it's not on a textured background, though.

Regards,

Richard.

Edited to add:
I have been talking to a colleague who has a personal copy of Photoshop. It appears one way (and it isn't the only way) to do editable mirrored text in Photoshop is to draw it on a layer and mirror the whole image.
This makes me think that I could draw the entire picture in memory, but mirrored, write the text the correct way round on it, find the pixels affected by the text, then mirror them back to the chart image way round & store those mirrored pixels for display. This would anti-alias the mirrored pixels properly and is only a step or two beyond what already happens in the program. I shall have to think on it and experiment - but after my 'interim' release.

[richardandtracy]

I have made the interim release with the image editor. I must admit that the help file is not adequate at the moment and I have not been able to incorporate rotation/mirroring for blocks. I have realised these need a re-jig of the data structure for blocks that will be quite far reaching and I won't be able to do it quickly.

So, to describe the image editor's features as the help file doesn't:

The current chart image is loaded into the image editor as a background layer that can be either visible or invisible. This chart image is not altered until you accept the edits & leave the image editor.
There are various entities that are drawn over the top of the background image to change the way the picture looks. These entities are:
Pencil Tool (colours individual pixels). Left click uses the Foreground Colour, right click the Background Colour.
Lines: Left click uses the Foreground Colour, right click the Background Colour.
Flood fills: Left click uses the Foreground Colour, right click the Background Colour.
Text: Foreground Colour only
Rectangles: Line around rectangle = Foreground Colour, Fill = Background Colour.
Ellipses/Circles: Line around entity = Foreground Colour, Fill = Background Colour.

There are two 'special' entities too.
Erase Item
Block.

The 'Erase Item' is like a pencil tool, but instead of using the standard colours, it takes the colour from the background image - so it looks like the items above the background have been erased.
The 'Block' is a useful entity in that it can be made from a segment of the image, and copied over the picture. Then, when it is changed, all linked blocks change to match the changed one. So if you have a border repeat around a picture, the single repeat can be changed and the whole border will change.

Blocks, Erase Items and Pencil tools can have their pixels individually edited
All entities can be copied and moved.
All entities except blocks and text can be mirror copied.
All entities except blocks can be rotated using onscreen picks. Additionally, text can be rotated in 0.1 degree increments.

To edit an entity, it has to be selected. This can be by means of an individual pick, or dragging a selection rectangle. If the rectangle is dragged from left to right, the entity has to be entirely within the rectangle. If dragged from right to left, part of the entity has to be in the rectangle.

Pencil tool colours can be changed.
Flood fills can be moved. If necessary they can be regenerated in the new position. Take an example, if initially filling a rectangle, it can be moved from inside to a coloured rectangle outside the rectangle. Then if re-generated it can change into a flood fill around the rectangle.
Text colour can be changed, as can the text angle and text itself. The text can be anti-aliased using DMC or Blended colours if required. The anti-alias settings can be changed and the text re-generated with the new settings.
Lines, Rectangles and Ellipses/Circles can have their line width changed, Fill colour changed (if appropriate) and be anti-aliased using Subsurface Sampling of 1(no anti-aliasing),2 or 4(best anti-aliasing, 16 samples used for each pixel). It should be noted that due to the limitations of the DMC colours available, anti-aliasing is not always successful, just making the lines wider and still blocky. If that is the case, select the item, change the anti-alias settings, and regenerate it. The ends of lines can be moved, as well as the mouse movements to change the size of rectangles & ellipses/circles.

All entities can be selected, deselected, shown & hidden. The display order can be shuffled about, so early items can be brought to the front etc.
There is the usual zooming, between a size of 1 and 30 screen pixels per stitch. There are ruler bars to the side and top. DMC and Blend palettes are available, and a click on one of the foreground/background colour boxes will open a small window to select a colour from direct RGB entry. The colour space division idea from the colour conversion window is used in this window during anti-aliasing and can be changed.

I think that's close to it. 13600 lines of code (approx. 200 pages of printout) make up this segment of the program.

After all that effort, I do hope someone finds it useful!


For the next stage I plan on adding polylines, filled polylines and rotation/mirroring of blocks and updating the help file.

Regards,

Richard.

[richardandtracy]

I have been fairly busy programming the image editor.

I have implemented open polylines, finished mirroring & rotating blocks and ironed a few bugs out - code length 16650 lines and exe size 2.7Mb. The help file is going to take a fair time to get to a satisfactory level of detail, but I think I know what I need to do with it. I shall be implementing filled, closed polyshapes before I release the program.

However, I have a couple of questions.

1. I had a thought about gradient fills. Would it be useful to specify the fill colour at all four corners of the bounding rectangle around a flood fill, and then apply a gradient within that area? Would dithering be useful to cover this area better?
2. Would it be useful to be able to change the size of the stitched area within the image editor (crop or expand, or both)?
3. At the moment, when you leave the image editor, everything is collapsed into a bitmap and all the vector stuff is lost. Would it be useful to keep it instead? This may be possible.
4. Are there any other standard shapes you'd like to see beyond a Rectangle or Ellipse/Circle?
5. Is there anything else you'd like to see in an image editor that you've never seen or not seen often enough?

Regards,

Richard.

[richardandtracy]

I think I have finished, barring final bug checks, the image editor and a somewhat re-vamped help file so that context sensitive help can be obtained in the 'Image Editor', 'Thread Conversion' and 'Backstitch over High Res Image' windows.
Want to have a play with it and create a nice Chatelaine type in the 'Image Editor' with it before release, just to show off its capabilities.

Both the .exe and .chm files are up to 2.7Mb each, which isn't small. In fact the image editor code length is now up to 18500 lines, and that's a fair sized program in its own right. I do hope this work will be useful to someone!

Regards,

Richard.

[SteveM]

Richard,

I stumbled across your program quite by accident while looking for various pattern making software that I could demo. To my knowledge yours is the only one I've been able to try that has a blended thread pattern built in, can apply dithering to just the selected area, has multiple dithering algorithms, and the merge/replace tool.

About four years ago I saw an acquaintance stitching the pattern below and inquired about it and found out that she had made the pattern from a photo using Pattern Maker. It is a nice pattern, but I would like to try other artwork so I've been evaluating various software to see which is the best at photo imports of the nautical type. In the series of pictures below are the best outputs that I have so far been able to get out of these programs with a limitation of about 150 colors. Dithering was typically used to get the best appearance from the sky. The photo and all generated patterns are 384x409. On the left is the "color block" view of each generated pattern and on the right the jpeg that was used to generate the patterns.


Reference Pattern Maker output:


MacStitch:


BlendThread solid colors:


BlendThread blended colors:


Ryijy:


PCStitch:


DP Cross Stitch Pro:


STOIK:



In my opinion Pattern Maker is the best over all with this photo with MacStitch a close second. (Update - Ryijy result was not accurate and has been updated). I don't know why, but this image caused some big name programs like PC Stitch to do a face plant and the output of WinStitch was not as good as MacStitch even though they are from the same company. I'm going to keep playing with your program, try turning down the green weighting and maybe do some targeted dithering.

Once I've had some more time to experiment I may have some suggestions. Any suggestions I make now would be out of ignorance as I've not done due diligence yet on some of the features. Thanks again for providing this free program to us all.

[richardandtracy]

I'm always interested to hear suggestions.
Just at the moment I don't have a great deal of time to incorporate changes - but if you suggest any, I am willing to consider them.

I have contacted Jari Haurito about Ryijy several times, and shown him links to teach himself dithering. I am fond of Ryijy as it was the program I used initially and we have corresponded a fair bit about printout methods. Anyway, I am actually quite impressed with Ryijy's conversion there.

Regards,

Richard.

[richardandtracy]

One other thing that I have not explained very well in the help file (or at all, really) is the effect of the weighting. If you want it more sensitive to errors in a particular colour, INCREASE the weighting for the colour not decrease it (as that will allow greater error tolerance).

Regards,

Richard

[SteveM]

Thanks, I'm going to try turning up the blue next time. I do like how your program has few to no restrictions and more control over the process so I'd like to get a handle on how to tune it to each photo. It is a bit slow and I have to ignore messages saying it is hung, but eventually it does complete which is good enough for me.

I went through your colour picker code to see how you process the image pixels and search for a match, interesting stuff. It looked like there was a lot of Borland specific calls in there, but I'm a bit rusty on C and haven't done much C++ and have don't remember much about what is in the standard libraries. However, I was still able to get the general idea.

[richardandtracy]

I am using Borland specific 'DynamicArrays' to a great extent to store data. These are arrays are useful because you don't have to decide in the initial programming stage how much memory to allocate, and it frees the memory completely when the array length is set to zero. I have never had a lesson in C++, and only 12hrs of being taught Pascal, so my programming is probably wildly inefficient.

I do use the Borland TBitmap and TImage classes a great deal - they surface the Windows API commands in an easy to use wrapper, and mean that I don't have to bother about a great deal of normal windows API programming. In fact the Borland stuff makes it possible for someone relatively comfortable in Sinclair Spectrum/Timex 2000 Basic to programme in windows. Just as well, as that's where I got my first programming experience.

I have written my own graphics library (GraphicsHelp.h/GraphicsHelp.cpp) to try to handle RGB components - and have tried to speed it up as much as I can - but with my limited programming experience I generally have to use the brute ignorance & force method. It came as a huge programming leap for me to use the left & right bit shift commands (>> and <<) to try to speed integer calculations up in the colour matching and dithering routines. Anyway - that's why it takes time to do the conversion, particularly on large images. If you can change the image size you want to convert, in an image editor, to the size you want to stitch, it will make a noticeable difference in the conversion times. Instead of a 1200 x 900, open a 400 x 300 pixel image. That really does save time.

Regards,

Richard.

[SteveM]

On reviewing my Ryijy results I determined that I failed to select the DMC colours in both places as needed and ended up with a fairly idealized colour pallet. The image above has been updated and Ryijy demoted.

The first photo that I tried to use was 1500x1063 so I had quite a bit of time to wait between operations and eventually got smart and started using a smaller test subject. My C tools of choice are vim and gcc so all of those nifty built in libraries that you use are certainly not something that I am used to seeing or using.

This evening I decided to try to make the sky a truer color of blue, but no matter what I did there was little change. I did some sampling of the sky in GIMP and got an average color in the cloud-free area near the top left and top right of RGB 101 158 224 and compared that to what most programs were representing it with, something like 107 158 191. Using your colour picker I typed in the desired RGB values and quickly realized the problem is that there are no DMC threads even close. So I added my sky colour to the dmc2a file and this is the result using about 150 solid colors with dithering:

BlendThread solid colors:


Not bad, especially if I was to spend a little more time tuning it with blended colours and limiting the dithering to the sky and water. At least it proves to me that the colour picking algorithm is not in error.

My first feature request is that when in the image editor and using "colour picker" to select a colour from the pattern I get a little box that lights up with the selected colour, but I don't get any indication of the coulor vitals such as DMC number and name. If I click on that coulored box at least I do see the RGB values.

-Steve

[richardandtracy]

Steve,

The dmc2a.csv file contains the rgb representations of thread colours, and is based on published data with the exception of the DMC beige swatch. [I discovered an error in the data for the darkest colour in the swatch, and it appeared that the published data was much too light for the darkest colour in the swatch. Both Jari and I adapted our DMC colour input data to reflect this.] I have plans, at some point, to stitch up a 10 x 10 block of the lightest and darkest colours in each swatch on the same piece of fabric and then photograph them at the same time in the same light conditions, so I can get my own data to correct all the other colours. It is something I'm not entirely looking forward to, which is why I've not done it yet.

To simply add the rgb colour to the thread colour data does not make a thread of that colour exist, even if the conversion is improved. If you have data reflecting commonly available thread colours/makes and are willing to share this data with program users, I'd be happy to add the rgb thread representations into the input data. If you have colour data on a non-DMC manufacturer's entire range, I may well have to re-jig the program so that the user can opt to select different thread palettes or simply select 'use the best'.

Regards,

Richard