Jul
22

Assembler, part 4 2017

What am I doing?

I've once again resumed working on the interface rather than the actual core; I still don't even know if any of this truly works. Sheesh.

Not a whole lot to report from yesterday as I practically spent the day reading while doing the odd bit of UI tinkering. Big thanks to Rob for reminding me about SyntaxBox! Scintilla is pretty horrible overall and SB is done so much better, so I'm glad the editor is now more solid. Code folding was extremely easy to implement, unlike Scintilla where I just couldn't be bothered and worked on another part instead.

Today I started a new control for displaying parsed instructions. Think of it as a ListBox but with no user interaction, and the "current item" is always vertically centred and highlighted; currently adding column support to tabulate the view. Oh, issue: I can't get the background transparent no-matter what. I've done millions of transparent controls in the past, so this is confusing the hell out of me.

Next, I need to have a go at actually executing instructions. I've done very little bit-work in .NET so I'm not sure how that's going to go, especially as I don't even know the syntax for VB.Net; presumably it's going to be bat-shit insane compared to C#.

Nothing instead of just null still annoys me to this very day.

Jul
20

Assembler, Part 3 2017

I've gone ahead and added the BaseMnemonic class, along with an IMnemonic interface that all CPU mnemonics (instructions, op-codes - whatever you want to call them) inherit from.

The past hour or so has been spent improving the interface and adding things where necessary. The main window now has the standard set of root menu items (File, Edit, View, Debug, Tools, Help) and there's a (currently collapsed) project viewer.

Added a Messages window that will show To-do items, syntax errors, and the like. Also added Status Register to the Registers window. The tooltip for each register's value also displays the value of that register's contents in base 10 (Decimal) and likely base 16 (Hex) in the near future, too.

Next up is making use of the BaseMnemonic class by implementing the Move instruction. Oooh!

Messages and Registers windows


Assembler, Part 2

Part number the two!

Did some more work on the Assembler project (part 1 here) that didn't just involve improving the interface. Ooh, ooh - what?

Parsing. That's what.

Hitting Run (or F5) now results in the entered code being parsed and added to an Instructions collection. The following style is correctly parsed:

<mnemonic><datatype> <source_operand>, <dest_operand>

An example of which is:

move.l #4, d0 ; an optional comment which isn't explicitly captured

mnemonic is the instruction (move, add, etc). datatype is the size which is either .b (byte), .w (word), .l (long-word). source_operand and dest_operand is where stuff happens (values, registers, addresses, and the like).

Each line in the source is read one-by-one. Once a line is read, a regular expression1 match is attempted (hard-coded add and move for now) .

(add|move)(?:\.(l|b|w)?)? *(.*), (\w*)

Note that this expression is far from complete. First change would involve removing the rigid white-space structure.

The reason for the datatype being optional in the regex is because I thought it could be omitted from assembler source. Doing a bit of research seems to show that everyone always includes a datatype (or size as I think is the nomenclature), so I've dropped the idea of having a default type. Still, this now means I can detect and specifically give an error about any missing types.

Anyway, back to parsing.

If the line fails the regex match, then onto the next line. If it succeeds, the parsing continues and an Instruction object is created containing mnemonic and datatype. I haven't yet figured out how the source and destination operands are going to be represented. In assembler code, they can be addresses, registers, or whatever else so I don't really know what's going to happen until I just start typing.

Undecided if mnemonics should inherit from a baseMnemonic and be individual classes, or if I should continue with the mnemonic simply being an enum on Instruction. I like instructions being self-contained, but comparisons become more involved. I'm leaning more towards baseMnemonic with a defined Interface so instructions can do whatever the hell they want as long as they take in an input and give back an output.

Thinking further ahead, I'm also undecided if the entire assembler source should be parsed in one go or instead proceed one line at at time. The latter will allow code to be edited while the program is running, which is quite nice, and goes hand-in-hand with the Step debugger feature.

Right now, I can't think of any advantages of parsing in one go (parse each line and populate Instructions immediately before execution even starts). Maybe "compiler" optimisations at a later date? Performance isn't even a minor consideration, so I don't see that as much of a win. I think the step-by-step method simply gives more advantages and so that may very well be the route to take.

1 I can highly recommend Regex Pal for writing and testing expressions.

Jul
19

Assembler, Part 1 2017

I don't yet have a name for the Assembler language program-thingy I'm doing so I'm just going to refer to it as Assembler (giev suggestions!) for the time being. Cool? Cool.

As mentioned in a previous post, I'm taking time out from my commercial software and am instead working on fun little projects. Clearly, our definition of the word fun differs greatly.

I wrote a Turing program a long time ago and recently it occurred to me that I could get a more advanced version by essentially emulating a CPU. The Turing machine is pretty much what that is. In this case, I'd be going with a Motorola 68000. Because it's lovely, and because it seems simple enough as compared to the nightmares that are the x86 and x68 architectures. Bear in mind that I don't really know much about assembler/assembly other than tinkerings back in the mid-to-late 1990s.

Most of the time so far has been spent creating a lexer (syntax highlighter) for Scintilla. Being a .NET OOP guy and working with Scintilla.NET (which doesn't attempt to really refactor anything), its API is a complete mess. So it took me quite a while to get highlighting happening, and it's still not done. Thankfully there's a project called ScintillaNET-Kitchen that helps out.

I haven't gone for a multi-document interface or anything that you'd find in more complete IDEs - how do I even know if this project will work satisfactorily? Hell, saying that, I've (partly!) made it possible to switch in new CPUs. I think that itself is a bit much and so don't currently plan on fully adding support for such a feature. Hmm... I could also create a chipset with multiple chips...? No - NO. Stop right there.

Yeah, alright - what the eff is the point?

Fun. As said above.

I'm going to keep adding to it. Forever. CPUs don't tend to do a whole lot on their own; their instruction sets aren't very big, either. Lots of shunting of data all over the place. Of course this is simplified and from someone that doesn't really assembler-me-do, but how hard can it be?

So, if I can get the basics working then I can start adding built-in routines that a program can call to do more advanced stuff. A video display would need to be added, of some sort. Only being able to manipulate registers isn't the most exciting thing after the initial novelty wears off.

Shut up and tell me if it's done yet

Hell no it isn't done. It doesn't even parse the entered code yet. There's the editor with all its syntax highlighting tomfoolery, and there's the Registers window that displays the contents of each register. That bit works; it shows exactly what's in the registers, but they can't be set using code yet.

There's a screenshot below. That's all there currently is. Will still if I have time to get the parsing - at least started - in today.

Assembler, screenshot uno

Edit: Did the opposite of what I said and have continued working on the interface. Oops.

Jul
17

A Week's Break 2017

The drawback of having a Category - Sub-category format is that you end up with silly stuff like the above.

Alright, I've just decided that I'm going to use the next week or so to work on fun little projects. Fun being an entirely subjective word, here.

I won't do anything on FileSieve, or Wake On LAN Ex, or anything of the sort. What I am going to work on, though, is... uh, I dunno - something?

One thing I thought of is y'know those Turing machines? The thing with the limited instruction set involving a pointer where you can program it to move to different cells and increment values, etc? No?

Well, one of those. I remember making one in the morning after skipping college (I was young and an idiot, as opposed to now where I'm just old and an idiot) and I found it weirdly enjoyable.

I'd like to do an advanced version that uses instructions. Like, my own assembly language, ya dig? The language and parsing, etc, can be easily fudged with regular expressions, and the main core can be done with a state machine.

I only used assembly back in the Amiga days in very limited ways, such as counting the number of carriage returns in a text file to determine how many pages a document has. (this is referencing The Word, #amigarulezok)

This Turing++ project will give me a better grasp of how the hell people can write full programs in such a language. Squillions upon squillions of lines of code for even the most basic of things - how do you keep track of it all?


New Sections on BootBlock.co.uk

When I moved this blog thing over from the sub-domain, I kinda forgot that the main site had various links on it to different places such as the forums, my software, and of course to contact me.

Had a bit of a WTF moment the other day as I realised all of the said links were gone! No-one could view anything other than the absolute crud that I dribble onto here; it was only supposed to be a sub-domain and not the actual, main site.

Well. That changed.

I've now added a few sections at the top-right of this page; the main ones are links to my free and commercial software, and an area that contains contact information.

And I've just realised that this site doesn't say who I actually am.

Jul
16

FileSieve 4.20, Scripting, and Threads 2017

That's a new version of FileSieve done and released. I think this particular version has the biggest change log so far. It would've been bigger - he said - but I, for some reason, decided to revisit the Script plugin.

I don't think I've gone into this before other than with a small forum post, but I started a new (Method) plugin a while ago that is what amounts to a small programming environment.

The purpose of the Script plugin is to allow the user to program their own Method and Modifier plugins. Full intellisense, code completion, debugging, and all that guff. It's undoubtedly far too much effort for the outcome of only a few people even being able to effectively use it as it requires programming knowledge. And then the documentation. Holy crap - no, I've already been through that nightmare with the original VoiceMate Professional.

While working on the prescanning threads feature, for no explicable reason I decided to jump into the Script plugin and work on that. The plugin all now builds and runs, but then common sense won out in the end and I unloaded the project from the solution and ended it there.

As for the prescan threading: it didn't make it into 4.20. The scanning was never designed from the beginning with concurrency and so random things aren't thread-safe, resulting in Weird Things Happening. I'm not going to delay the new big release because of this, so the threading features were disabled and the new release has been made.

Jul
15

Pre-scanning Threads 2017

Just a quick one today as I'm feeling like sheeeeeet.

FileSieve 4.20 is shaping up to be a pretty hefty release as it contains the most changes for the 4 series to date. Because of that, I'm adding more changes, and even more changes. Gonna break a record? Do it properly.

Today's (partial, as I was away for a lot of the day) task is to try and get the Prescanning split into multiple threads.

Before FileSieve does its sieving thing, it has a Prescan stage where it gathers information on all of the files to be processed. This is currently a sequential process and can take a while as it needs to traverse the file system and grab various pieces of metadata.

Now, what if the files reside on different drives? We can portion out all Source Items (a directory, essentially) onto different threads depending on which drive they reside on. This needs a bit more than simply creating a list of Source Items per drive letter due to logical and physical drives. That is, a single drive may have multiple logical partitions, and we don't want to thrash the disk because we're too dumb to realise all of those partitions share the same drive heads.

This Prescan threading feature has been sat dormant in FileSieve for quite a while as I didn't get around to implementing the are-drive-letters-on-the-same-drive logic. Over the next couple of days I'm going to see if I can determine how feasible this is; I definitely do not want to rely on Windows Management Instrumentation as the WMI infrastructure is one bloaty mutha - it takes literal seconds to instantiate for the first time.

Jul
13

Wake On LAN Ex 3.10 2017

Another day, another release.

There was only really supposed to be one change to this new release:

Performance improvements to the loading and saving of settings.

This is the result of the FileSieve and Newtonsoft JSON.Net findings. I reworked my (JSON/XML) serialisation code and took out the options that allowed you to specify Image and SecureString type conversion, and now instead require that all properties of those types have a TypeConverter attribute specified.

<JsonConverter(GetType(SecureStringConverter))>
Public Property Password As SecureString
    ...
End Property

As well as simplifying the code - we're only talking about a reduction of 4 lines here - it removes the enormous performance drain of JSON.Net's TypeConverter CanConvert property. It does mean you can't just pass the relevant converter option, but with the huge gains - who cares, really?

Sorry, I lost the main point of this point: there are now quite a few changes in this new WoLEX release; once I started, I couldn't stop.

Jul
12

FileSieve and Newtonsoft JSON.NET 2017

After the last Wake On LAN Ex release, FileSieve needs to be sorted out and released itself. This particular 4.20 release contains significant bug fixes that really needs to be put in the hands of FS users.

But when - just as before - I was just about to get the remaining issues fixed, I ran into another issue. A gigantic, showstopping issue that I'm absolutely confounded by.

Any time FileSieve does any (de)serialisation, JSON.NET throws exceptions; specifically, the .NET runtime throws exceptions. FileSieve worked absolutely fine forever - until this started happening yesterday. All of my other software that also (de)serialise are doing the same thing.

The code in-question (VB.NET):

Return JsonConvert.DeserializeObject(Of T)(tr.ReadToEnd, jss)

The exception is all of the following:

  • System.NotSupportedException: 'TypeConverter cannot convert from (null).'
  • System.NotSupportedException: 'TypeConverter cannot convert from System.String.'
  • System.NotSupportedException: 'CollectionConverter cannot convert from (null).'
  • System.NotSupportedException: 'CollectionConverter cannot convert from System.String.'

The error isn't technically being thrown by JSON.NET itself, but by .NET and so consequently JSON.NET isn't providing me with any useful information on what the underlying cause is.

I'd post an example class that's being exceptioned on, but they're a bit too involved. But don't forget: they were working completely fine previously.

I didn't make any changes to any of the classes being serialised, so why has this started happening? Who the hell knows. What I do know is that all development has halted.

This is with Newtonsoft JSON.NET 10.0.3.

Edit: It appears I've found one of the causes. It's when using the property attribute DefaultValue. In this case, it's when setting the default value for SystemColors. Here's the code below, in this one particular case.

    Private _customColor As Color = SystemColors.Info

    <Category(""), Description(""), Browsable(True), DefaultValue(GetType(SystemColors), "Info")>
    Public Property CustomColor() As Color
        Get
            Return Me._customColor
        End Get
        Set(ByVal value As Color)
            Me._customColor = value
        End Set
    End Property

Edit 2: Managed to fix all of the FileSieve issues; this means I have a ton of other projects I need to similarly fix. I hope it's just a bug introduced into JSON.NET that stops DefaultValue() from being used. Each complex property now needs a corresponding method that determines if it should be serialised or not.

To fix the above color property, remove the DefaultValue attribute from the property and use the following method within the same class as the property itself:

    ''' <summary>Determines if CustomColor should be serialised depending on if it's the same colour as its default value.</summary>
    ''' <returns><c>True</c> if CustomColor is different to the default and should be serialised, otherwise <c>False</c>.</returns>
    <EditorBrowsable(EditorBrowsableState.Never), Browsable(False)>
    Public Function ShouldSerializeCustomColor() As Boolean
        Return (Not Me._customColor.ToArgb.Equals(SystemColors.Info.ToArgb))
    End Function

The reason for the ToArgb() is as follows:

To compare colors based solely on their ARGB values, you should use the ToArgb method. This is because the Equals and Equality members determine equivalency using more than just the ARGB value of the colors. For example, Black and FromArgb(0,0,0) are not considered equal, since Black is a named color and FromArgb(0,0,0) is not.