Threaded Events

Got a good idea? You can suggest new features here.
Post Reply

Would you like to see EventGhost have the ability to parallel process events?

Yes
4
80%
No
1
20%
I have not clue what you are mumbling about
0
No votes
 
Total votes: 5

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Threaded Events

Post by kgschlosser » Sat Dec 07, 2019 11:29 am

I decided since i have much more coding experience under my belt that I would have a crack at adding the Threaded Events to EG once again.
I do have to say that this time it is a whole lot better and a HUGE reduction in the code. no API got broken at all it is still 100% in tact.

How the Threaded events work is when an event gets triggered the program check to see if the event has been triggered before. if it has not a new event gets created.. The event is the core of the threaded events. it is the thread worker. that event will exist as long as EG is running. The thread of the event will be started as needed. there is a queue for payloads. this queue gets used if the same event comes in bunch of times in succession we do not want to have the same macro getting processed in parallel. the results of that could be UGLY!. But we can process 2 different macros at the same time. and that is what this does..

Here is an example. If you have a remote control and the events that get produced are enduring events. if you create a loop to say increase the volume while the button is being held down on the remote. No other events will get triggered while EG is in this busy state. For me that is a big issue as I have a movie collection that has several thousand flicks and i do hold that up or down arrows to scroll though the list.. and not having other events getting processed when there are 3 other remotes and TV's attached to the system is not a cool thing. This solves that problem.

The threading bits are done. I am going to have to recode a large portion of the GUI specifically the log and the system tray icon. this is because e still want to keep some kind of an alignment with what event is running what macro/actions. and having a bunch of events running at the same time the log is going to get messy. I know how I am going to go about coding this portion of it. I am also going to add a dialog that will show the events that are running and ones that are idle.

I am also going to have to rework the clear pending events action. so a specific event can be targeted or all events can be.

I wanted to take a poll and see how many people would be interested in having this feature.
If you like the work I have been doing then feel free to Image

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Sat Dec 07, 2019 11:33 am

I wanted to mention that I added this ability with only 320 lines of code!!!!! my last crack at it I think I was around 1500 lines of code
If you like the work I have been doing then feel free to Image

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Sat Dec 07, 2019 3:12 pm

ok so I have some metrics for ya.

51 threads generating 200 events each thread. the total run time to generate the 10200 events. 3.276 seconds!
doing the same test without the threaded events. 4.384 seconds.

Now I know that it doesn't seem to be much. it's a 33% increase in speed. which is actually pretty decent in it's self.. But there is a constant that is causing a slow down. the log. rendering that icon and text takes the exact same amount of time per item either way no way to speed that up.

so here are some numbers with the logging removed.
threaded .921 seconds
non threaded 2.464

Now there is the real jump. 167.53% increase in speed!!!!!

so for those of you that have multiple remotes and you have a button being held down on each remote that causes a stream of events.. This would be a huge improvement.

I am sure I will be able to squeeze more performance out of it.. actually.. I am going to be able to because of how I am going to handle the logging. the events will not always be seen in the log window. only the first occurrence of the event will be. then when you do a hover over that event a balloon will open showing all of the logging that has taken place for that event.

So said and done I should be somewhere close to 376% faster then the non threaded events. YEE HAW!!!
If you like the work I have been doing then feel free to Image

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Sat Dec 07, 2019 3:13 pm

If you vote No it would be nice to know what the reasoning is.
If you like the work I have been doing then feel free to Image

Medy
Experienced User
Posts: 86
Joined: Sat Nov 15, 2014 8:33 pm
Location: Germany

Re: Threaded Events

Post by Medy » Sun Dec 08, 2019 2:22 am

I dont know if there are many ppl out there that use Eg to that extent.

Though if you want to do it , why not :D...

.. if it will be ultimately worth the time / enjoyment ...
and if doesnt seem to big of a deal to implement such a thing .. why even ask ? ... I recon it would be quite a mammoth task ... where you have to plug alot of holes down the road ... or maybe not `?

This is off topic but & shameless plug :
Have you taken a look at my feature request for broadlink? I would like to work on plugins for things that I use ..
The problem is I bumb into restriction since Eg is running older python version & it seems hard to get anything to run that has dependencies...

and it sucks to ask for help everytime ... just because there is some import issues or whatever where I dont even now what the problem is.

maybe this would benefit EG software as well .. if for development one could make use of libraries more easiliy ?
Just wild assumptions though :D
... so basicly a major overhaul ?

Take this with a grain of salt ..I might be a odd one wanting eventghost to support everything xD

viewtopic.php?f=5&t=10403

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Sun Jan 19, 2020 5:47 am

I am going to dive into updating plugins to be python 3.x compatible I want to have all of the plugins converted before a release of EG takes place.

It someone is familiar with PHP and would be willing to help me add a plug-in repository to the website that would be a great help. I am PHP ignorant, the code is already written and it attaches to phpbb but there are a coupl of things that do not work properly like the search in the forum will not search for content in the extension. Same thing goes vice versa. The extension has an issue tracker for each plug-in that has been uploaded to it. I want to be able to link that tracker to the forum thread for the plug-in
If you like the work I have been doing then feel free to Image

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Sun Jan 19, 2020 5:55 am

As far as the threaded events are concerned. Anyone that has a looping macro will benefit greatly from this change. The looping macros are commonly used when holding a remote button down. Normally while the macro is looping EG is unable to process any events. So if there are 2 remotes being used with EG and someone presses a button on the remote. Nothing is going to happen. So the likely hood of a button getting pressed a slew of times and other buttons getting pressed is high. These events would get queued so once the loop has finished the queued events would get processed and this can cause the system to go bonkers.

So by having the threaded events a macro could loop until the end of time and not stop other events from being processed.
If you like the work I have been doing then feel free to Image

Medy
Experienced User
Posts: 86
Joined: Sat Nov 15, 2014 8:33 pm
Location: Germany

Re: Threaded Events

Post by Medy » Tue Jan 21, 2020 4:43 pm

Ok. Keep in mind that im commenting from a perspective of someone who doesnt do all this ;).
And probablly cant even see the benefits at this point.

Dont let that stop you.
Im just commenting to give feedback ;). Kinda to represent a regular user :D

Essentially just wanted to help make you ,, make your decision if its worthwhile and if there even are ppl that look for this.


The one thing that I think about when putting in effort into smth is , if its worth the effort ... or if im getting some value out of it.

By me commenting I just wanted to add data ;) ...
Because nothing is worse than spending hundreds of hours .. if
A - you didnt enjoy it
B - no one values it :D

So in a sense by commenting , my intentnion was to dircet / help you ... a eventghost developer ... to things that make eventghost thrive.

My impression is that from being here , the participation is abit lacking.
Wich to me is a sign that there is not enough new ppl are aware of eventghost .. even though its like the best thing out here
in terms out automation / makros ... etc...

I havent think about it that far .. but essentially I would think that
efforts should be made towards making eventghost appeal more to regular users.

Im thinking about the most basic functionality of doing some simple stuff
with macros and hotkeys .... to capture audience attention.

You know I was feeling like doing magic when first creating my macros.
Gotta spread the magic to the web 2.0

so when you asking about your suggestion ... I thought ... who is this going to appeal to.
Rather have you focus on smth new then smth that further develop the hole in the graveyard.


Im using iobroker for home automation and love how simple it is with the web interface...
Abit of coding knowledege seems required ...
Eventghost is still more "regular joe" beginner friendly ...

Would be cool if eventghost would move into
the webspace .. appeal to "regular joe" ... and maybe interact with these tools ...
there seems to be tons of development.... gotta get into that space ... to benefit / grow through it..



Just throwing thoughts around.
Im basicly learning design / UI atm ...
Sign me up if you decide to go that route someday :D ... maybe I could contribute in the future ;) ...
Its probably a task for at least a small team ... so idk

STill love Eg.



Have a nice day

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Wed Jan 22, 2020 1:41 am

I get ya.. I am just responding to what you are saying...

This is what people want. they want smart home software where you have to pick up a phone and press a button to turn a light on. People like you and I do not need a GUI to turn a light on.. EG is home automation software the point to it is turn the light on automatically where no user involvement is needed. People seem to like the web GUI's and the apps that have lots of useless junk in them. things that are not really needed. Here is a great example. people want to see what the electric usage is.. so they will be able to lower it.. my thought process is who cares about having a graph when you can have automatic routines take place that will minimize the consumption for you.

we have about 400 unique downloads a week. people are finding it without an issue. The question is this.. are they using it? or do they look at it and say nahhh.. I know it has a real simple UI one could say because the lack of buttons and things to click on may cause people to think that it cannot do all that much..

or maybe people are using it and are not having any issues and do not feel the need to post in the forum..

This is the single largest question I have no answer to.

I want to pitch this at you..I am sure you have probably heard of Home Assistant. the #1 complaint and gripe is how slow the thing is. well... it is slow because it has a single event loop. It is the same EG in that respect.. But how the even loop functions is far different. Home Assistant goes and asks each and every single item to do an update. this is a dumb design because it is asking for update when the returned data is the same as what it already has. it is a complete waste.. and what makes it worse is the more you add the slower it becomes.

Eventghosts event loop sits and waits for something to tell it that a change has been made. and when that takes place it does what it needs to do. but it is still only able to do one thing at a time. I can loosen that restriction.

the other thing is people seem to want to control every aspect of their home from a raspberry pi, this is a joke to try and accomplish but that is what I am seeing.
If you like the work I have been doing then feel free to Image

Medy
Experienced User
Posts: 86
Joined: Sat Nov 15, 2014 8:33 pm
Location: Germany

Re: Threaded Events

Post by Medy » Wed Jan 22, 2020 7:14 pm

I image there are tons of ppl happyliy using Eg and never visit the forum.


Personally I would like to see the kind of engagement / support from the home automation space
move into eventghost.


Im kind of ok with using 2 programs ... iobroker + eventghost . Each does what they are good at.

Just feel like eventghost is missing out by not being a part of the rise of home automation as much.

As you stated they are proabbly less experienced when it comes do doing it right. But they are where
the hpye is at .. and thriving.

I just wondered if there isnt any opportunty for EG to step in ;).

User avatar
kgschlosser
Site Admin
Posts: 5144
Joined: Fri Jun 05, 2015 5:43 am
Location: Rocky Mountains, Colorado USA

Re: Threaded Events

Post by kgschlosser » Thu Jan 23, 2020 8:20 am

I am not sure how to go about tackling that. I believe that EG is not as main stream because it is not cross platform and because it has no "fluffy" web GUI.

EG is a dual purpose automation tool. It can automate just about any device that has the ability to communicate. But it is also able to control the PC it is installed on. the last bit is what makes EG not able to be cross platform. Maybe this is something I should dive into. EG can run as a service I have done it. I have run it without a GUI and used a websocket connection to see what was going on. Plugins that rely on using Windows notifications would no longer work. a windows service is not allow to interact with a desktop. This functionality can be achieved by a program that would get run and would sit in the task tray once a user has logged in. so this is not to much of an issue to overcome. It can actually be extended so that more then a single client computer could have that application running and could connect to EG, Macros can be created to target controlling a specific computer.

in order to make EG cross platform all of the Windows specific bits would need to be stripped out and replaced by modules that would be loaded depending on the OS it was installed on. I do wish I knew someone that knew how to code in Python and is familiar with Linux API. It would make things a whole lot easier.
If you like the work I have been doing then feel free to Image

Post Reply