Chat/GPT-4 released

sad
1684646979529.png
 
The article was probably hard to write, considering that like it say:
The judge ruled that developers could bypass Apple's commission by sending users to a web browser to make purchases. OpenAI is not doing that, perhaps because the process is more clunky than Apple's built-in payment processor.

They seem to consider there is a lot of value of letting the store be in charge (for a lot of openAI product you have to buy your subscription and enter your key manually, they probably feel it hurt adoption a lot), it is not like they are forced to pay via the store, it is just quite the moat for the companies to make people go around.
 
LukeTbk said:
The article was probably hard to write, considering that like it say:
The judge ruled that developers could bypass Apple's commission by sending users to a web browser to make purchases. OpenAI is not doing that, perhaps because the process is more clunky than Apple's built-in payment processor.

They seem to consider there is a lot of value of letting the store be in charge (for a lot of openAI product you have to buy your subscription and enter your key manually, they probably feel it hurt adoption a lot), it is not like they are forced to pay via the store, it is just quite the moat for the companies to make people go around.
Click to expand...

OpenAI Considers Exit From Europe - Faces Planned Legislation from Regulators​

by T0@st Today, 12:29 Discuss (8 Comments)
OpenAI's CEO, Sam Altman, is currently exploring the UK and Europe on a PR-related "mini" world tour, and protesters have been following these proceedings with much interest. UK news outlets have reported that a demonstration took place outside of a university building in London yesterday, where the UCL Events organization hosted Altman as part of a fireside discussion about the benefits and problems relating to advanced AI systems. Attendees noted that Altman expressed optimism about AI's potential for the creation of more jobs and reduction in inequality - despite calls for a major pause on development. He also visited 10 Downing Street during the British leg of his PR journey - alongside other AI company leaders - to talk about potential risks (originating from his industry) with the UK's prime minister. Discussed topics were reported to include national security, existential threats and disinformation.
 
Have you guys found free chatGPT has become useless? Last few times I asked it to create some code or test templates it just gave up half way through and crashed.
 
serpretetsky said:
Have you guys found free chatGPT has become useless? Last few times I asked it to create some code or test templates it just gave up half way through and crashed.
Click to expand...
I felt a change, but maybe it is because I have read it elsewhere or maybe it is at the time it was used, maybe using less compute to make room for the smarthphone App traffic ?

The paid a la co-pilotX type seem fine.
 
  • Like
Reactions: erek
like this
LukeTbk said:
I felt a change, but maybe it is because I have read it elsewhere or maybe it is at the time it was used, maybe using less compute to make room for the smarthphone App traffic ?

The paid a la co-pilotX type seem fine.
Click to expand...
I don't have access to paid co-pilot x, but the regular version I had to shut off it's so bad. GPT-4 my boss pays for and is a good tool.
 
  • Like
Reactions: erek
like this
serpretetsky said:
Have you guys found free chatGPT has become useless? Last few times I asked it to create some code or test templates it just gave up half way through and crashed.
Click to expand...
Yeah 3.5 is bad at coding. 4 is a very significant upgrade.
 
  • Like
Reactions: erek
like this
Decko87 said:
Yeah 3.5 is bad at coding. 4 is a very significant upgrade.
Click to expand...
No that's not what I meant. I already kind of know the limits of 3.5 for my application. What i mean is that now it just seems to crash on me before even finishing it's response. Before it would respond with bad solutions, now it crashes halfway through its bad solution.
 
  • Like
Reactions: erek
like this
erek said:
"JPMorgan is developing a ChatGPT-like A.I. service that gives investment advice" https://www.cnbc.com/2023/05/25/jpmorgan-develops-ai-investment-advisor.html
Click to expand...

Lawyer 'Greatly Regrets' Relying on ChatGPT After Filing Motion Citing Six Non-Existent Cases (reason.com)71​

Posted by EditorDavid on Saturday May 27, 2023 @06:34PM from the disorder-in-the-court dept.
The judge's opinion noted the plaintiff's counsel had submitted a motion to dismiss "replete with citations to non-existent cases... Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations... The bogus 'Varghese' decision contains internal citations and quotes, which, in turn, are non-existent."

Eugene Volokh's legal blog describes what happened next:Thursday, plaintiff's counsel filed an affidavit in response, explaining that he was relying on the work of another lawyer at his firm, and the other lawyer (who had 30 years of practice experience) also filed an affidavit, explaining that he was relying on ChatGPT... ("The citations and opinions in question were provided by Chat GPT which also provided its legal source and assured the reliability of its content...")
Their affidavit said ChatGPT had "revealed itself to be unreliable,"
 
In an opinion piece for the New York Times, Vox co-founder Ezra Klein worries that early AI systems "will do more to distract and entertain than to focus." (Since they tend to "hallucinate" inaccuracies, and may first be relegated to areas "where reliability isn't a concern" like videogames, song mash-ups, children's shows, and "bespoke" images.) https://entertainment.slashdot.org/...e-matrix-theory-of-ai-assisted-human-learning
 
erek said:
In an opinion piece for the New York Times, Vox co-founder Ezra Klein worries that early AI systems "will do more to distract and entertain than to focus." (Since they tend to "hallucinate" inaccuracies, and may first be relegated to areas "where reliability isn't a concern" like videogames, song mash-ups, children's shows, and "bespoke" images.) https://entertainment.slashdot.org/...e-matrix-theory-of-ai-assisted-human-learning
Click to expand...
AI writing assistants can cause biased thinking in their users https://arstechnica.com/science/202...nts-can-cause-biased-thinking-in-their-users/
 
erek said:

Lawyer 'Greatly Regrets' Relying on ChatGPT After Filing Motion Citing Six Non-Existent Cases (reason.com)71​

Posted by EditorDavid on Saturday May 27, 2023 @06:34PM from the disorder-in-the-court dept.
The judge's opinion noted the plaintiff's counsel had submitted a motion to dismiss "replete with citations to non-existent cases... Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations... The bogus 'Varghese' decision contains internal citations and quotes, which, in turn, are non-existent."

Eugene Volokh's legal blog describes what happened next:Thursday, plaintiff's counsel filed an affidavit in response, explaining that he was relying on the work of another lawyer at his firm, and the other lawyer (who had 30 years of practice experience) also filed an affidavit, explaining that he was relying on ChatGPT... ("The citations and opinions in question were provided by Chat GPT which also provided its legal source and assured the reliability of its content...")
Their affidavit said ChatGPT had "revealed itself to be unreliable,"
Click to expand...
 
Just a heads-up but ChatGPT 4.0 using a WolframAlpha plugin for math is absolutely bonkers in what it can do. I'm chewing through molar conversions like it's nothing, and it handles volumetrics very well. It's not been perfect, but if I request that it pulls publications to confirm molar conversion ratios it will catch itself if using incorrect balance. Not totally self sufficient yet, but terrifying on how close it is.
 
Version 4 is only available after getting the "Plus" upgrade, right? Any free previews or something?
 
LukeTbk said:
The article was probably hard to write, considering that like it say:
The judge ruled that developers could bypass Apple's commission by sending users to a web browser to make purchases. OpenAI is not doing that, perhaps because the process is more clunky than Apple's built-in payment processor.

They seem to consider there is a lot of value of letting the store be in charge (for a lot of openAI product you have to buy your subscription and enter your key manually, they probably feel it hurt adoption a lot), it is not like they are forced to pay via the store, it is just quite the moat for the companies to make people go around.
Click to expand...
They aren't doing it for a few reasons, but real-world examples of where this is happening the overhead ends up adding a lot of work to the developers while saving very little in comparison.
Visa, Mastercard, Paypal, etc... are gonna get their cut one way or another and small developers don't have the bargaining power of Apple to negotiate fees so they get hit comparatively harder, then they also need to deal with hosting their own payment portal, dealing with fraud cases (stolen/fake cards), call centers for customer complaints, additional accounting requirements, blah blah blah, what they have found over and over again is they save 3-5% by taking on a lot of the work themselves, but if they need to contract out for customer complaints, or payment gateway hosting then the savings shrinks to be about the same. They add some flexibility for the customers as they can have different payment options than just Apple Pay (Paypal, debit, alternate cash apps, gift cards, etc) but they add that flexibility by adding an inconvenience, which results in lower sales as it gives the potential customer enough time to rethink their $1.99 purchase and often results in a "bah this is too much work, I didn't really need that anyways" type moment along the way.
 
ChatGPT Plus now has a new python interpretor mode

- File attachment upload support (e.g. CSV, XLS, TXT, XML, etc)
- Self-executing & self-debugging (it interprets the Python error messages and modifies the Python code -- to varying success levels)
- Can generate resulting PNG files (or copy and paste CSV / SQL / TXT etc) in Python processing the file attachment (e.g. charts, graphs, etc) displayed right on the spot.
- You can even ask it to make changes (fonts, colors, paddings, sizes, layout).

Version 4.0 with the new sandboxed Python interpretor/self-debugging mode (Settings -> Beta) in can generate chart .png files now. For example I can upload an .xls file to ChatGPT now, ask it to reformat the data, and then generate a custom graph. It's not perfect.

It even debugs its own Python programs (not always perfectly) but it's neat to see it do 3 edits to its own programming based on the Python interpretor error messages.

Usually it will succeed most simple requests, but be prepared for a bit of prompt engineering (e.g. it failed on certain chart-reformatting tasks so you may have to ask it to "rewrite the python program from scratch" and iterate from that).

It's exactly what I've been predicting -- AIs that can debug its own programming mistakes.

It can't yet (interpret) the resulting .PNG file for errors, but it certainly does things like pie charts, GANTT charts, scatterdot graphs, and other simple graph images generated by Python programming that you can copy and paste to fix final errors, if you'd like. It will fix unstructured data if you ask it to, and generate appropriate custom regex expressions, or simply natural-language-modify the data before giving it to the interpretor ("Re-summarize the Notes field to no more than 80 characters. Use common shorthand like "KB" instead of "kilobytes").

So you can natural-language reprocess the data before letting the Python process the data. Things like "I've attached a spreadsheet. In unstructured column 3, please change all positive responses to "Yes" and negative responses to "No". In the final Notes column, please resummarize long notes to anything no more than 80 characters. Sort the rows based on the sense of urgency written in the Description field, from most urgent to least urgent. Do this before providing the data into the Python program I ask you to write to generate a custom chart."

It lets you upload messy junk .xls / .csv / .txt / etc to this new ChatGPT Python Interpretor beta. Multiple attachments if you want (up to 25MB total). And asking it to process the data ("Column 3 has a 3-letter substring somewhere near the end that represents calendar month, please find a way to intelligently parse that out that won't create false positives") and it'll figure out its own what kind of filtering is best, whether regexing or via another filtering method (including some techniques of AI-modifying the data before being processed by Python). And you can do natural language asks ("Also the last column is multiline notes that sometimes mentions urgency. For those, plot those specific graph bars in red color code.").

Or you can ask it to create cross-references like a relational database (.SQL format) out of unstructured spreadsheets that refers to each other a little too casually. Be careful what you do with it though, e.g. pasting it into MySQL to create multiple tables (like loading a .sql backup)

For minor errors, I can just tell it natural language to modify the Python program that generates the .png graph
("That's too small. I need at least 1920x1080 or bigger. Also, make sure there are at least 30 pixels of padding around the edge of the image file. And change the graph grid to 4-pixel dotted blue, and graph boundary to 4-pixel solid red, and I want the graph title to be in 36-point Arial Bold please.") and it self-modifies the Python program, re-runs it, and displays the .png file right inside the ChatGPT 4.0 chat.

Pretty neat. It's still Wright Brothers era, but after only 5 minutes of prompt engineering, I have nice graph .png files that would otherwise have taken me 20-30 minutes to create elsewhere, more if I needed to manually filter the data (e.g. parsing unformatted data). I do have to proofread the work and doublecheck the results -- but it is a net timesaer.

It's not going to always be useful, but a big timesaver when I need to process random generic data (that doesn't have security considerations) into a Python program that it writes, and then self-executes to output a .png file that I can save from the browser to disk.

Almost scary to see ChatGPT Plus subscription now writing, executing and debugging its own Python programs. It's a heavily sandboxed interpretor, but, that's where SkyNet lore begins.

Not going to use ChatGPT to generate public content, but it's a good data processor assistant for things like internal data analysis. Like processing data into graphs now -- as long as you don't get too complex (e.g. creating complex organizational charts or flowcharts that it can't self-debug -- and it screwed up the arrow positions when drawing since it can't 'see' the result of the graphs it drew).

Metaphorically, AI 'assistants' are a good help especially if you are scatterbrained about your written notes (forgetting to reorganize your notes). During my last convention trip, I copy and pasted raw notes of what I did/learned at the most important booths, gave it my goals, and asked it to output a .csv spreadsheet of priority clients to followup. It was 90% accurate on prioritizing for me -- it correctly got the sense from my raw notes. As I acknowledge I am not stellar on time management tasks, this is one area where AI is already starting to help out on, as long as I get decent enough at correctly prompt-engineering as I am at bash-scripting.

It's still a bunch of unconnected/incomplete small proto-AGI islands (I would like it to be able to interpret images already, and be able to self-debug its own image-creation glitches, including graphics glitches in chart .png files). It's easy to be unable to figure out what AI is good for on the fly as you do your workday, because of all the gaps. But it's now getting far more helpful than say, Siri, for actually doing things like creating little Python programs that is designed to process csv/xls/txt into png (graphs/charts/etc), and I can modify the Python myself to finish off a final occasional remaining chart glitch faster than ChatGPT can.

Regardless, always double check works -- always. At least the paid 4.0 output is easier to double check than the 3.5 speil.

50% of the problem is correctly realizing (early enough) I'm working on a task that could save time with a little AI boost --
There are lots of mini timesaver tasks that in retrospect, realize I could now have saved, for mudane things like manually reformatting more 'casual' data that's too time consuming to write a one-time disposable Python program for. Since it can do the 'data pre-cleanup' in an AI-way (even recognizing and fixing CSV or XML errors, like unclosed tags, or unclosed quotation marks between commas) before the structured Python processing.

It's easy to ignore the AI since it's imperfect, so part of the skill is recognizing what it does well at, and what it does terribly at, and knowing how to take-turns cleaning each other up (e.g. letting it do 90% of the work generating a TODO list from raw meeting minutes transcript, and I do the final 10% cleanup like a couple of forgotten bullets).

Never depend on the AI. But find ways to make it an assistant for you, in a way that is easy to doublecheck its work -- so you can use/discard -- depending on the quality of the data.

They can be massive timesavers for mudane reformatting/parsing notes/organizing/prioritizing from unorganized raw copy and pasted text (or the new file attachment upload feature in the Python Interpretor Beta enabled in Settings).

Mind you...

I am looking for multiple 'good' AI suppliers, as I don't want ChatGPT to have the monopoly, and I'm concerned about some of the 'ethics' of AI training too, as per media reports of OpenAI questionable AI-training practices -- that does give me pause for concern. But, I'm not an ostrich head in the ground. I need to understand how to operate AI's at least.

EDIT: IMPORTANT -- If you use AI to cleanup your CSV/XLS data, do it in two separate tasks. Ask it to do a CSV cleanup first, then ask it to generate the PNG. First, upload the file attachment but ask it to NOT produce a Python program, and simply ask it to output a new copy-paste cleaned-up CSV with all the AI autocorrection, be very specific you want to use its own AI for autocorrection, rather than Python for autocorrection.
Only by doing it this way, I was able to convert this Excel to CSV to PNG using ChatGPT using a two-step method, where I saved the interim CSV to reupload as a new file attachment (or ask it to assign to a string variable in a Python program)

This was an educational training exercise spreadsheet, aimed at helping me learn how to use AI to reprocess messy data.

1689474214838.png


- Left label color is whether or not "Complete" (aka purchased)
- Bar color is corrected "Category" color
- Bar length is total cost (unit times cost)
- Paddings / leading / trailing "garbage" was removed by AI upon my request during AI-cleanup task
- Casual category was autocorrected / remapped by AI upon my request to a boolean red/green for label color.
- Casual complete was autocorrected / remapped by AI upon my request to a bar color.

bargraph.png


To generate the PNG (which could be downloaded directly from ChatGPT plus), ChatGPT wrote this final Python program:

Code: 
import io
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
from matplotlib import colors

# Load the cleaned data from the CSV string
cleaned_csv_string = 'Item,Category,Complete,Units,Credits,Notes,Location\nTHX-1138  rivet,Red,True,2,70,Purchased in the back alley of Mos Eisley,Mos Eisley\nTurbo crystal,Red,True,1,124,From our friend at Jedi temple on Coruscant,Coruscant\nResistance sticker,Green,True,2,23,Tourist gift shop at Coruscant starship terminal,Coruscant\nDarksaber polish,Green,False,1,90,?,Unknown\nBantha paint,Red,False,1,55,You don\'t want to know,Unknown\nFlux capacitor,Green,True,1,198,Classified. (From another galaxy far far away),Other Galaxy\nFlap for speeder,Yellow,True,20,3,Scrap from storeroom,Storeroom\nHyperspace O-Ring,Red,False,2,30,Purchased in the back alley of Mos Eisley 5 rotations ago,Mos Eisley\nIG-11 replacement toe,Green,False,10,14,Imperial junkyard,Imperial Junkyard\nLightsaber,Red,True,1,100,From our friend at Jedi temple on Coruscant,Coruscant\nDroid bolts,Red,False,8,22,Jawas\' native homeworld. We forgot what planet it was,Unknown\nServo,Red,True,1,99,Mos Eisley,Mos Eisley\nBB-8 lens cleaner,Green,False,1,123,Storeroom,Storeroom\nXP-38 Speeder,Yellow,False,1,341,Unknown,Unknown\nBeskar (0.001 porg),Green,True,1,250,From our contact at Mandalore,Mandalore\nTatooine sand bricks,Yellow,True,120,1,From Jawa\'s homeworld planet,Jawa\'s Homeworld\n'
df_cleaned = pd.read_csv(io.StringIO(cleaned_csv_string))

# Convert Units and Credits to numeric values and compute total cost
df_cleaned['Units'] = pd.to_numeric(df_cleaned['Units'], errors='coerce')
df_cleaned['Credits'] = pd.to_numeric(df_cleaned['Credits'], errors='coerce')
df_cleaned['Total Cost'] = df_cleaned['Units'] * df_cleaned['Credits']

# Map colors to hexadecimal color codes for the graph
color_dict = {
    'Red': '#FF0000',
    'Green': '#008000',
    'Yellow': '#FFFF00'
}

# Map completion status to colors for the labels
complete_color_dict = {
    True: 'green',
    False: 'red'
}

# Create a new figure
plt.figure(figsize=(19.2, 10.8), dpi=100)

# Create a horizontal bar plot with color-coded bars
sns.barplot(data=df_cleaned, y='Item', x='Total Cost', palette=df_cleaned['Category'].map(color_dict), edgecolor='blue', linewidth=4)

# Color-code the y-axis labels according to the completion status
[plt.gca().get_yticklabels()[i].set_color(complete_color_dict[df_cleaned['Complete'].iloc[i]]) for i in range(len(df_cleaned))]

# Set the background color to white
plt.gca().set_facecolor('white')

# Set the title and labels
plt.title('Cost per Item', fontsize=24, pad=20, fontname='Sans-serif')
plt.xlabel('Total Cost', fontsize=18, labelpad=20, fontname='Sans-serif')
plt.ylabel('Item', fontsize=18, labelpad=20, fontname='Sans-serif')

# Increase the size of the y-axis labels
plt.yticks(fontsize=16)

# Add a border around the plot
plt.gca().spines['top'].set_visible(True)
plt.gca().spines['top'].set_linewidth(4)
plt.gca().spines['bottom'].set_visible(True)
plt.gca().spines['bottom'].set_linewidth(4)
plt.gca().spines['right'].set_visible(True)
plt.gca().spines['right'].set_linewidth(4)
plt.gca().spines['left'].set_visible(True)
plt.gca().spines['left'].set_linewidth(4)

# Add padding around the plot
plt.tight_layout(pad=5)

# Save the plot as a PNG file
plt.savefig('/mnt/data/bar_graph.png')
I made no programming corrections, although for the first-time-around, had to tell AI some graph glitch errors until it was fixed.

Except for two nebulous implied locations (Trashosha and Tatooine as "trick locations") it even correctly parsed an AI-guessed location for a new "Location" field, but I forgot to ask it to display the data. (Ah well -- but you can view the cleaned_csv variable, that it correctly pulled things like "Mos Eisley" and "Imperial Junkyard" for a newly created "Location" column.
It goofed up a few requests, like more padding around the edges (something it has a hard time doing, but one can easily do it manually by editing the Python)

Code: 
cleaned_csv_string = 'Item,Category,Complete,Units,Credits,Notes,Location\nTHX-1138  rivet,Red,True,2,70,Purchased in the back alley of Mos Eisley,Mos Eisley\nTurbo crystal,Red,True,1,124,From our friend at Jedi temple on Coruscant,Coruscant\nResistance sticker,Green,True,2,23,Tourist gift shop at Coruscant starship terminal,Coruscant\nDarksaber polish,Green,False,1,90,?,Unknown\nBantha paint,Red,False,1,55,You don\'t want to know,Unknown\nFlux capacitor,Green,True,1,198,Classified. (From another galaxy far far away),Other Galaxy\nFlap for speeder,Yellow,True,20,3,Scrap from storeroom,Storeroom\nHyperspace O-Ring,Red,False,2,30,Purchased in the back alley of Mos Eisley 5 rotations ago,Mos Eisley\nIG-11 replacement toe,Green,False,10,14,Imperial junkyard,Imperial Junkyard\nLightsaber,Red,True,1,100,From our friend at Jedi temple on Coruscant,Coruscant\nDroid bolts,Red,False,8,22,Jawas\' native homeworld. We forgot what planet it was,Unknown\nServo,Red,True,1,99,Mos Eisley,Mos Eisley\nBB-8 lens cleaner,Green,False,1,123,Storeroom,Storeroom\nXP-38 Speeder,Yellow,False,1,341,Unknown,Unknown\nBeskar (0.001 porg),Green,True,1,250,From our contact at Mandalore,Mandalore\nTatooine sand bricks,Yellow,True,120,1,From Jawa\'s homeworld planet,Jawa\'s Homeworld\n'

This completed an educational-test of ChatGPT Plus;
 
Last edited:
Edited my above post with images. Learning how to prompt engineer correctly took me about 30-40 minutes, but now I can do it in about 15 or less. It will screw things up if you try to ask it to do too many tasks at a time. It took roughly 5 Python self-debug attempts before it worked, though first time took about 30 (as I figured out how to prompt engineer it correctly). Break it down to a data cleanup task, followed by a chart generation task.

Learning prompt engineering quirks ("correctly wording your request to ChatGPT") is not too different from learning how to use a scientific calculator for the first-ever time, you get faster at prompt engineering. THEN it finally begins to save you time, as long as you double check data by viewing the XLS + PNG side by side. Things like that. And you can always revise/rerun the Python locally too!

I am looking for multiple 'good' AI suppliers, as I don't want ChatGPT to have the monopoly, and I'm concerned about some of the 'ethics' of AI training too, as per media reports of OpenAI questionable AI-training practices -- that does give me pause for concern. This is still Wild West.

The GPT 3.5 speil is not useful to me, but the 4.0 (paid) is now "useful enough".

But, I'm not an ostrich head in the ground. I need to understand how to operate AI's at least.
 
Last edited:
Chief Blur Buster said:
ChatGPT Plus now has a new python interpretor mode

- File attachment upload support (e.g. CSV, XLS, TXT, XML, etc)
- Self-executing & self-debugging (it interprets the Python error messages and modifies the Python code -- to varying success levels)
- Can generate resulting PNG files (or copy and paste CSV / SQL / TXT etc) in Python processing the file attachment (e.g. charts, graphs, etc) displayed right on the spot.
- You can even ask it to make changes (fonts, colors, paddings, sizes, layout).

Version 4.0 with the new sandboxed Python interpretor/self-debugging mode (Settings -> Beta) in can generate chart .png files now. For example I can upload an .xls file to ChatGPT now, ask it to reformat the data, and then generate a custom graph. It's not perfect.

It even debugs its own Python programs (not always perfectly) but it's neat to see it do 3 edits to its own programming based on the Python interpretor error messages.

Usually it will succeed most simple requests, but be prepared for a bit of prompt engineering (e.g. it failed on certain chart-reformatting tasks so you may have to ask it to "rewrite the python program from scratch" and iterate from that).

It's exactly what I've been predicting -- AIs that can debug its own programming mistakes.

It can't yet (interpret) the resulting .PNG file for errors, but it certainly does things like pie charts, GANTT charts, scatterdot graphs, and other simple graph images generated by Python programming that you can copy and paste to fix final errors, if you'd like. It will fix unstructured data if you ask it to, and generate appropriate custom regex expressions, or simply natural-language-modify the data before giving it to the interpretor ("Re-summarize the Notes field to no more than 80 characters. Use common shorthand like "KB" instead of "kilobytes").

So you can natural-language reprocess the data before letting the Python process the data. Things like "I've attached a spreadsheet. In unstructured column 3, please change all positive responses to "Yes" and negative responses to "No". In the final Notes column, please resummarize long notes to anything no more than 80 characters. Sort the rows based on the sense of urgency written in the Description field, from most urgent to least urgent. Do this before providing the data into the Python program I ask you to write to generate a custom chart."

It lets you upload messy junk .xls / .csv / .txt / etc to this new ChatGPT Python Interpretor beta. Multiple attachments if you want (up to 25MB total). And asking it to process the data ("Column 3 has a 3-letter substring somewhere near the end that represents calendar month, please find a way to intelligently parse that out that won't create false positives") and it'll figure out its own what kind of filtering is best, whether regexing or via another filtering method (including some techniques of AI-modifying the data before being processed by Python). And you can do natural language asks ("Also the last column is multiline notes that sometimes mentions urgency. For those, plot those specific graph bars in red color code.").

Or you can ask it to create cross-references like a relational database (.SQL format) out of unstructured spreadsheets that refers to each other a little too casually. Be careful what you do with it though, e.g. pasting it into MySQL to create multiple tables (like loading a .sql backup)

For minor errors, I can just tell it natural language to modify the Python program that generates the .png graph
("That's too small. I need at least 1920x1080 or bigger. Also, make sure there are at least 30 pixels of padding around the edge of the image file. And change the graph grid to 4-pixel dotted blue, and graph boundary to 4-pixel solid red, and I want the graph title to be in 36-point Arial Bold please.") and it self-modifies the Python program, re-runs it, and displays the .png file right inside the ChatGPT 4.0 chat.

Pretty neat. It's still Wright Brothers era, but after only 5 minutes of prompt engineering, I have nice graph .png files that would otherwise have taken me 20-30 minutes to create elsewhere, more if I needed to manually filter the data (e.g. parsing unformatted data). I do have to proofread the work and doublecheck the results -- but it is a net timesaer.

It's not going to always be useful, but a big timesaver when I need to process random generic data (that doesn't have security considerations) into a Python program that it writes, and then self-executes to output a .png file that I can save from the browser to disk.

Almost scary to see ChatGPT Plus subscription now writing, executing and debugging its own Python programs. It's a heavily sandboxed interpretor, but, that's where SkyNet lore begins.

Not going to use ChatGPT to generate public content, but it's a good data processor assistant for things like internal data analysis. Like processing data into graphs now -- as long as you don't get too complex (e.g. creating complex organizational charts or flowcharts that it can't self-debug -- and it screwed up the arrow positions when drawing since it can't 'see' the result of the graphs it drew).

Metaphorically, AI 'assistants' are a good help especially if you are scatterbrained about your written notes (forgetting to reorganize your notes). During my last convention trip, I copy and pasted raw notes of what I did/learned at the most important booths, gave it my goals, and asked it to output a .csv spreadsheet of priority clients to followup. It was 90% accurate on prioritizing for me -- it correctly got the sense from my raw notes. As I acknowledge I am not stellar on time management tasks, this is one area where AI is already starting to help out on, as long as I get decent enough at correctly prompt-engineering as I am at bash-scripting.

It's still a bunch of unconnected/incomplete small proto-AGI islands (I would like it to be able to interpret images already, and be able to self-debug its own image-creation glitches, including graphics glitches in chart .png files). It's easy to be unable to figure out what AI is good for on the fly as you do your workday, because of all the gaps. But it's now getting far more helpful than say, Siri, for actually doing things like creating little Python programs that is designed to process csv/xls/txt into png (graphs/charts/etc), and I can modify the Python myself to finish off a final occasional remaining chart glitch faster than ChatGPT can.

Regardless, always double check works -- always. At least the paid 4.0 output is easier to double check than the 3.5 speil.

50% of the problem is correctly realizing (early enough) I'm working on a task that could save time with a little AI boost --
There are lots of mini timesaver tasks that in retrospect, realize I could now have saved, for mudane things like manually reformatting more 'casual' data that's too time consuming to write a one-time disposable Python program for. Since it can do the 'data pre-cleanup' in an AI-way (even recognizing and fixing CSV or XML errors, like unclosed tags, or unclosed quotation marks between commas) before the structured Python processing.

It's easy to ignore the AI since it's imperfect, so part of the skill is recognizing what it does well at, and what it does terribly at, and knowing how to take-turns cleaning each other up (e.g. letting it do 90% of the work generating a TODO list from raw meeting minutes transcript, and I do the final 10% cleanup like a couple of forgotten bullets).

Never depend on the AI. But find ways to make it an assistant for you, in a way that is easy to doublecheck its work -- so you can use/discard -- depending on the quality of the data.

They can be massive timesavers for mudane reformatting/parsing notes/organizing/prioritizing from unorganized raw copy and pasted text (or the new file attachment upload feature in the Python Interpretor Beta enabled in Settings).

Mind you...

I am looking for multiple 'good' AI suppliers, as I don't want ChatGPT to have the monopoly, and I'm concerned about some of the 'ethics' of AI training too, as per media reports of OpenAI questionable AI-training practices -- that does give me pause for concern. But, I'm not an ostrich head in the ground. I need to understand how to operate AI's at least.

EDIT: IMPORTANT -- If you use AI to cleanup your CSV/XLS data, do it in two separate tasks. Ask it to do a CSV cleanup first, then ask it to generate the PNG. First, upload the file attachment but ask it to NOT produce a Python program, and simply ask it to output a new copy-paste cleaned-up CSV with all the AI autocorrection, be very specific you want to use its own AI for autocorrection, rather than Python for autocorrection.
Only by doing it this way, I was able to convert this Excel to CSV to PNG using ChatGPT using a two-step method, where I saved the interim CSV to reupload as a new file attachment (or ask it to assign to a string variable in a Python program)

This was an educational training exercise spreadsheet, aimed at helping me learn how to use AI to reprocess messy data.

View attachment 583602

- Left label color is whether or not "Complete" (aka purchased)
- Bar color is corrected "Category" color
- Bar length is total cost (unit times cost)
- Paddings / leading / trailing "garbage" was removed by AI upon my request during AI-cleanup task
- Casual category was autocorrected / remapped by AI upon my request to a boolean red/green for label color.
- Casual complete was autocorrected / remapped by AI upon my request to a bar color.

View attachment 583604

To generate the PNG (which could be downloaded directly from ChatGPT plus), ChatGPT wrote this final Python program:

Code: 
import io
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
from matplotlib import colors

# Load the cleaned data from the CSV string
cleaned_csv_string = 'Item,Category,Complete,Units,Credits,Notes,Location\nTHX-1138  rivet,Red,True,2,70,Purchased in the back alley of Mos Eisley,Mos Eisley\nTurbo crystal,Red,True,1,124,From our friend at Jedi temple on Coruscant,Coruscant\nResistance sticker,Green,True,2,23,Tourist gift shop at Coruscant starship terminal,Coruscant\nDarksaber polish,Green,False,1,90,?,Unknown\nBantha paint,Red,False,1,55,You don\'t want to know,Unknown\nFlux capacitor,Green,True,1,198,Classified. (From another galaxy far far away),Other Galaxy\nFlap for speeder,Yellow,True,20,3,Scrap from storeroom,Storeroom\nHyperspace O-Ring,Red,False,2,30,Purchased in the back alley of Mos Eisley 5 rotations ago,Mos Eisley\nIG-11 replacement toe,Green,False,10,14,Imperial junkyard,Imperial Junkyard\nLightsaber,Red,True,1,100,From our friend at Jedi temple on Coruscant,Coruscant\nDroid bolts,Red,False,8,22,Jawas\' native homeworld. We forgot what planet it was,Unknown\nServo,Red,True,1,99,Mos Eisley,Mos Eisley\nBB-8 lens cleaner,Green,False,1,123,Storeroom,Storeroom\nXP-38 Speeder,Yellow,False,1,341,Unknown,Unknown\nBeskar (0.001 porg),Green,True,1,250,From our contact at Mandalore,Mandalore\nTatooine sand bricks,Yellow,True,120,1,From Jawa\'s homeworld planet,Jawa\'s Homeworld\n'
df_cleaned = pd.read_csv(io.StringIO(cleaned_csv_string))

# Convert Units and Credits to numeric values and compute total cost
df_cleaned['Units'] = pd.to_numeric(df_cleaned['Units'], errors='coerce')
df_cleaned['Credits'] = pd.to_numeric(df_cleaned['Credits'], errors='coerce')
df_cleaned['Total Cost'] = df_cleaned['Units'] * df_cleaned['Credits']

# Map colors to hexadecimal color codes for the graph
color_dict = {
    'Red': '#FF0000',
    'Green': '#008000',
    'Yellow': '#FFFF00'
}

# Map completion status to colors for the labels
complete_color_dict = {
    True: 'green',
    False: 'red'
}

# Create a new figure
plt.figure(figsize=(19.2, 10.8), dpi=100)

# Create a horizontal bar plot with color-coded bars
sns.barplot(data=df_cleaned, y='Item', x='Total Cost', palette=df_cleaned['Category'].map(color_dict), edgecolor='blue', linewidth=4)

# Color-code the y-axis labels according to the completion status
[plt.gca().get_yticklabels()[i].set_color(complete_color_dict[df_cleaned['Complete'].iloc[i]]) for i in range(len(df_cleaned))]

# Set the background color to white
plt.gca().set_facecolor('white')

# Set the title and labels
plt.title('Cost per Item', fontsize=24, pad=20, fontname='Sans-serif')
plt.xlabel('Total Cost', fontsize=18, labelpad=20, fontname='Sans-serif')
plt.ylabel('Item', fontsize=18, labelpad=20, fontname='Sans-serif')

# Increase the size of the y-axis labels
plt.yticks(fontsize=16)

# Add a border around the plot
plt.gca().spines['top'].set_visible(True)
plt.gca().spines['top'].set_linewidth(4)
plt.gca().spines['bottom'].set_visible(True)
plt.gca().spines['bottom'].set_linewidth(4)
plt.gca().spines['right'].set_visible(True)
plt.gca().spines['right'].set_linewidth(4)
plt.gca().spines['left'].set_visible(True)
plt.gca().spines['left'].set_linewidth(4)

# Add padding around the plot
plt.tight_layout(pad=5)

# Save the plot as a PNG file
plt.savefig('/mnt/data/bar_graph.png')
I made no programming corrections, although for the first-time-around, had to tell AI some graph glitch errors until it was fixed.

Except for two nebulous implied locations (Trashosha and Tatooine as "trick locations") it even correctly parsed an AI-guessed location for a new "Location" field, but I forgot to ask it to display the data. (Ah well -- but you can view the cleaned_csv variable, that it correctly pulled things like "Mos Eisley" and "Imperial Junkyard" for a newly created "Location" column.
It goofed up a few requests, like more padding around the edges (something it has a hard time doing, but one can easily do it manually by editing the Python)

Code: 
cleaned_csv_string = 'Item,Category,Complete,Units,Credits,Notes,Location\nTHX-1138  rivet,Red,True,2,70,Purchased in the back alley of Mos Eisley,Mos Eisley\nTurbo crystal,Red,True,1,124,From our friend at Jedi temple on Coruscant,Coruscant\nResistance sticker,Green,True,2,23,Tourist gift shop at Coruscant starship terminal,Coruscant\nDarksaber polish,Green,False,1,90,?,Unknown\nBantha paint,Red,False,1,55,You don\'t want to know,Unknown\nFlux capacitor,Green,True,1,198,Classified. (From another galaxy far far away),Other Galaxy\nFlap for speeder,Yellow,True,20,3,Scrap from storeroom,Storeroom\nHyperspace O-Ring,Red,False,2,30,Purchased in the back alley of Mos Eisley 5 rotations ago,Mos Eisley\nIG-11 replacement toe,Green,False,10,14,Imperial junkyard,Imperial Junkyard\nLightsaber,Red,True,1,100,From our friend at Jedi temple on Coruscant,Coruscant\nDroid bolts,Red,False,8,22,Jawas\' native homeworld. We forgot what planet it was,Unknown\nServo,Red,True,1,99,Mos Eisley,Mos Eisley\nBB-8 lens cleaner,Green,False,1,123,Storeroom,Storeroom\nXP-38 Speeder,Yellow,False,1,341,Unknown,Unknown\nBeskar (0.001 porg),Green,True,1,250,From our contact at Mandalore,Mandalore\nTatooine sand bricks,Yellow,True,120,1,From Jawa\'s homeworld planet,Jawa\'s Homeworld\n'

This completed an educational-test of ChatGPT Plus;
Click to expand...
Chief Blur Buster said:
Edited my above post with images. Learning how to prompt engineer correctly took me about 30-40 minutes, but now I can do it in about 15 or less. It will screw things up if you try to ask it to do too many tasks at a time. It took roughly 5 Python self-debug attempts before it worked, though first time took about 30 (as I figured out how to prompt engineer it correctly). Break it down to a data cleanup task, followed by a chart generation task.

Learning prompt engineering quirks ("correctly wording your request to ChatGPT") is not too different from learning how to use a scientific calculator for the first-ever time, you get faster at prompt engineering. THEN it finally begins to save you time, as long as you double check data by viewing the XLS + PNG side by side. Things like that. And you can always revise/rerun the Python locally too!

I am looking for multiple 'good' AI suppliers, as I don't want ChatGPT to have the monopoly, and I'm concerned about some of the 'ethics' of AI training too, as per media reports of OpenAI questionable AI-training practices -- that does give me pause for concern. This is still Wild West.

The GPT 3.5 speil is not useful to me, but the 4.0 (paid) is now "useful enough".

But, I'm not an ostrich head in the ground. I need to understand how to operate AI's at least.
Click to expand...
Thanks bud, for the evaluation
 
Chief Blur Buster said:
Edited my above post with images. Learning how to prompt engineer correctly took me about 30-40 minutes, but now I can do it in about 15 or less. It will screw things up if you try to ask it to do too many tasks at a time. It took roughly 5 Python self-debug attempts before it worked, though first time took about 30 (as I figured out how to prompt engineer it correctly). Break it down to a data cleanup task, followed by a chart generation task.

Learning prompt engineering quirks ("correctly wording your request to ChatGPT") is not too different from learning how to use a scientific calculator for the first-ever time, you get faster at prompt engineering. THEN it finally begins to save you time, as long as you double check data by viewing the XLS + PNG side by side. Things like that. And you can always revise/rerun the Python locally too!

I am looking for multiple 'good' AI suppliers, as I don't want ChatGPT to have the monopoly, and I'm concerned about some of the 'ethics' of AI training too, as per media reports of OpenAI questionable AI-training practices -- that does give me pause for concern. This is still Wild West.

The GPT 3.5 speil is not useful to me, but the 4.0 (paid) is now "useful enough".

But, I'm not an ostrich head in the ground. I need to understand how to operate AI's at least.
Click to expand...

I felt the same way about 3.5. 4.0, with the Wolfram plugin, is useful enough now. I also use it to condense meeting minutes, format project proposals and quick draft emails where I only have to write a couple broken sentences and it formats and expands to something suitable.

I'm very interested in the python version being able to import data tables from PDFs and maybe do some simple statistics between data sets, find commonalities and generating the bones of graphs and visual data. If you have any prompts that you like that can do that, feel free to share. I would love to be able to easily generate relation plots with data sets.

edit - never mind...it just ate up a few tables I threw at it and automatically generated a correlation matrix between data items when requested to find a relationship

{{1.00, 0.83, 0.61, 0.88, 0.94, 0.69, NA, 0.77},
{0.83, 1.00, NA, NA, NA, NA, NA, 0.63},
{0.61, NA, 1.00, NA, NA, NA, NA, 0.56},
{0.88, NA, NA, 1.00, NA, NA, NA, 0.69},
{0.94, NA, NA, NA, 1.00, NA, NA, 0.83},
{0.69, NA, NA, NA, NA, 1.00, NA, 0.70},
{NA, NA, NA, NA, NA, NA, NA, NA},
{0.77, 0.63, 0.56, 0.69, 0.83, 0.70, NA, 1.00}}
Click to expand...

Each row and column corresponds to a different element concentration. The value at the intersection of a row and column is the correlation coefficient between those two elements. A correlation coefficient close to 1 or -1 indicates a strong positive or negative relationship, respectively, while a value close to 0 indicates a weak or no relationship.

Please note that some values are marked as NA (Not Available) because the calculation could not be performed due to missing data.

For example, the correlation between the first and second element concentrations is 0.83, indicating a strong positive relationship. On the other hand, the correlation between the first and third element concentrations is 0.61, indicating a moderate positive relationship.

Please let me know if you need further clarification or if you have any other questions.

Based on the correlation matrix, the following pairs of element groups have a strong correlation (values close to 1):

1. First and Second elements (correlation coefficient: 0.83)
2. First and Fourth elements (correlation coefficient: 0.88)
3. First and Fifth elements (correlation coefficient: 0.94)
4. First and Eighth elements (correlation coefficient: 0.77)
5. Second and Eighth elements (correlation coefficient: 0.63)
6. Third and Eighth elements (correlation coefficient: 0.56)
7. Fourth and Eighth elements (correlation coefficient: 0.69)
8. Fifth and Eighth elements (correlation coefficient: 0.83)
9. Sixth and Eighth elements (correlation coefficient: 0.70)

Please note that the element groups are referred to by their order in the data you provided, and the correlation coefficient is a measure of the strength and direction of the linear relationship between the two variables. A value close to 1 indicates a strong positive relationship.

Also, please note that some correlations could not be calculated due to missing data, and these are marked as NA (Not Available) in the matrix.
 
Last edited:
SilverSliver said:
I felt the same way about 3.5. 4.0, with the Wolfram plugin, is useful enough now. I also use it to condense meeting minutes, format project proposals and quick draft emails where I only have to write a couple broken sentences and it formats and expands to something suitable.
Click to expand...
Sounds like a great use case to produce responses to RFI's that have close due dates.
 
Hilarious, gave me instructions to install python, install the right libraries, configured the data to export and how to run the python script to generate the PNG for scatter plot graphs based on the data.

This is terrifying.
 
The Lurker said:
Sounds like a great use case to produce responses to RFI's that have close due dates.
Click to expand...
Yes, easiest if PDF is text based, where the data can be pulled and reprocessed.

For image based, try to get it to use Python OCR libraries to convert image based text in PDF into CSV and proces that data. Pretty neat if it can do that. Failing that (not installed in interpreter) it probably can help you with a python to OCR locally.

Just make sure you double check your AI assistant work. Never, ever skip that step. Include double check time in whether you decide AI saves you time.

EDIT: I don't create AI content. All my articles and posts are human written. I'm using AI for various internal tasks like data-processing, a StackOverflow companion (e.g. answering programming questions), and GitHub CoPilot style work (improved by the Genie plugin for Visual Studio Code + GPT 4.0 API + ability to ask it adhoc questions about my code), and various mudane tasks such as a format conversion or AI-based syntax checker (UBB,Markdown,HTML,XML,JSON,etc) which 4.0 is good at ("...You have an unclosed bracket at what seems to belong at...") finding things that other syntax checkers missed/redherringed/wildgoosed.
 
Last edited:
Chief Blur Buster said:
Yes, easiest if PDF is text based, where the data can be pulled and reprocessed.

For image based, try to get it to use Python OCR libraries to convert image based text in PDF into CSV and proces that data. Pretty neat if it can do that. Failing that (not installed in interpreter) it probably can help you with a python to OCR locally.

Just make sure you double check your AI assistant work. Never, ever skip that step. Include double check time in whether you decide AI saves you time.
Click to expand...

And now we can put AI prompt engineer on our LinkedIn.
 
Chief Blur Buster said:
Yes, easiest if PDF is text based, where the data can be pulled and reprocessed.

For image based, try to get it to use Python OCR libraries to convert image based text in PDF into CSV and proces that data. Pretty neat if it can do that. Failing that (not installed in interpreter) it probably can help you with a python to OCR locally.

Just make sure you double check your AI assistant work. Never, ever skip that step. Include double check time in whether you decide AI saves you time.
Click to expand...

Right! Gotta make it look a little bit personalized.
 
Chief Blur Buster said:
- Self-executing & self-debugging (it interprets the Python error messages and modifies the Python code -- to varying success levels)
Click to expand...
Once a system at google-microsoft-whoever get good enough on powerful enough hardware to

- make some code
- compile
- Look and score the result (did it compile, how much time it took to compile, did the code do what I wanted, what does it do otherwise, how fast the code run, interpret the UI-command line result)

And build its own home made datase and not just opensource code to train with, it could get really interesting not just it could explode the dataset quality-quantity it train with and its metric, but also use a lot of an human would have thought about it this way solution
 
The Lurker said:
Right! Gotta make it look a little bit personalized.
Click to expand...
Or accidental hallucinations. Learning what tasks it practically never hallucinates on, and what tasks has a hallucination-error on. One early pass in prompt engineering accidentally generated some "sample data" (to educate me as an aside) when I asked it too casually about how to do Task X vs Y, and the sample data propagated onwards to my next question that did not explicitly refer to my original data instead of the sample data. Keeping a wandering GPT4 "on rails" is extremely important --

Always interrupt GPT4 if it starts going off rails (GPT-4 is much slower than GPT 3.5) to save time -- and then "EDIT" your previous question with improved prompt engineering. It's like pressing a REWIND button on an assistant, to right before you even asked an earlier mis-worded question.

It's GIGO, so expert prompt engineering = big time saver.

If you run out of your 25 queries per 3 hours, get pay-per-use at platform.openai.com/playground which gives you a pay-per-use GPT 4 API instead and you have more unlimited queries in the Playground (up to a cost cap per month you specify). Or use a local GPT4 app with the API key. It's paid by processing time, so you can get billed up to your monthly payment cap. I also have that too, so I'm no longer constrained by the 25 query limit, and it's also cheaper-per-query average too. You may be waitlisted for access to the GPT 4 API instead of GPT-3.5 but, once you are onboard, you're far more capable than you'd be with just ChatGPT Plus paid subscription. However, I keep a concurrent ChatGPT Plus subscription for my light duty stuff, like:

For example, the paid "4" (unlike Bing's "4 Lite") is an absurdly fantastic regex expression generator, for example ("Give me a grep regex that can do...") for a disposable command line, like searching difficult-to-parse stuff in a giant logfile. I know my regex, but takes long time to create and debug complex nested expressions can take more than 15 minutes, when I get one in a mere 30 seconds or less from GPT 4+ ... the type of regex that's over 100 characters long.

1689554187390.png


Instead of two regexes, it combined it into one nested regex! (Where's Waldo: spot the pipe symbol inside the long regex)

Then once running and checking the results, I can even paste it back to the bot, and ask it to create a series of command lines that deletes all the spamers one by one. That's the kind of AI that can be a great tool at, in the fight against AI-based spambots that sometimes surge through weak filters that used to work well but before new better filters come out. Cat n mouse game. This sort of random mudane task.

It will not always save you time, but freeing up my "dreg" time to focus on more important tasks for my human brain -- that's worth the price
 
Last edited:
You must log in or register to reply here.
Back
Top